4

我已经使用 Apache Spark 部署了 Spring Boot 服务器，并且一切运行稳定。但是http://XXXX:4040/executors/ SparkUI executors端点抛出java.io.FileNotFoundException并且找不到/opt/x/x!/BOOT-INF/lib/spark-core_2.11-2.2.0.jar. 我检查了内罐。这个问题只发生在 Linux 上，在 Windows 上它可以正常工作。

2019-04-23 07:01:24,038 WARN [org.spark_project.jetty.servlet.ServletHandler] [SparkUI-36] - 
org.spark_project.jetty.servlet.ServletHolder$1: org.glassfish.jersey.server.internal.scanning.ResourceFinderException: java.io.FileNotFound
Exception: /opt/x/x.jar!/BOOT-INF/lib/spark-core_2.11-2.2.0.jar (No such file or directory)
        at org.spark_project.jetty.servlet.ServletHolder.makeUnavailable(ServletHolder.java:594)
        at org.spark_project.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:658)
        at org.spark_project.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:496)
        at org.spark_project.jetty.servlet.ServletHolder.ensureInstance(ServletHolder.java:788)
        at org.spark_project.jetty.servlet.ServletHolder.prepare(ServletHolder.java:773)
        at org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:578)
        at org.spark_project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
        at org.spark_project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
        at org.spark_project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
        at org.spark_project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.spark_project.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:461)
        at org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
        at org.spark_project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
        at org.spark_project.jetty.server.Server.handle(Server.java:524)
        at org.spark_project.jetty.server.HttpChannel.handle(HttpChannel.java:319)
        at org.spark_project.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)
        at org.spark_project.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
        at org.spark_project.jetty.io.FillInterest.fillable(FillInterest.java:95)
        at org.spark_project.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
        at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
        at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
        at org.spark_project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.spark_project.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.glassfish.jersey.server.internal.scanning.ResourceFinderException: java.io.FileNotFoundException: /opt/x/x.jar!/BOOT-INF/lib/spark-core_2.11-2.2.0.jar (No such file or directory)
        at org.glassfish.jersey.server.internal.scanning.JarZipSchemeResourceFinderFactory.create(JarZipSchemeResourceFinderFactory.java:90)
4

1 回答 1

3

问题的原因是 Jersey 的限制——它无法处理嵌套的 JAR 文件。您需要将 Spring Boot 配置为spark-core在您的应用启动时自动解压缩任何包含资源的 JAR。就我而言（我正在使用spark-core_2.12），解决方案是将以下部分添加到pom.xml文件中：

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <configuration>
        <requiresUnpack>
            <dependency>
                <groupId>org.apache.spark</groupId>
                <artifactId>spark-core_2.12</artifactId>
            </dependency>
        </requiresUnpack>
    </configuration>
</plugin>

一些资料来源：这里和这里

于 2019-07-28T09:19:41.273 回答