1

我正在尝试在 IntelliJ IDEA 中运行苏打水深度学习演示代码链接为：

https://github.com/h2oai/sparkling-water/blob/RELEASE-2.0.3/examples/src/main/scala/org/apache/spark/examples/h2o/DeepLearningDemo.scala

如果启动失败，异常是：

17/01/06 11:18:41 ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerJobEnd(1,1483672721446,JobFailed(org.apache.spark.SparkException: Job 1 cancelled because SparkContext was shut down))
    at org.apache.spark.util.SparkShutdownHookManager.runAll(ShutdownHookManager.scala:187)
    at org.apache.spark.util.SparkShutdownHookManager$$anon$2.run(ShutdownHookManager.scala:177)
    at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
    at java.lang.Thread.getStackTrace(Thread.java:1108)
    at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:632)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:1890)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:1903)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:1916)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:1930)
    at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:912)
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
    at org.apache.spark.rdd.RDD.withScope(RDD.scala:358)
    at org.apache.spark.rdd.RDD.collect(RDD.scala:911)
    at org.apache.spark.h2o.backends.internal.InternalBackendUtils$class.startH2O(InternalBackendUtils.scala:163)
    at org.apache.spark.h2o.backends.internal.InternalBackendUtils$.startH2O(InternalBackendUtils.scala:262)
    at org.apache.spark.h2o.backends.internal.InternalH2OBackend.init(InternalH2OBackend.scala:99)
    at org.apache.spark.h2o.H2OContext.init(H2OContext.scala:102)
    at org.apache.spark.h2o.H2OContext$.getOrCreate(H2OContext.scala:279)
    at org.apache.spark.h2o.H2OContext$.getOrCreate(H2OContext.scala:301)
    at com.xyz.HelloSparklingWater$.main(HelloSparklingWater.scala:35)
    at com.xyz.HelloSparklingWater.main(HelloSparklingWater.scala)

H2OContext构造and时看起来会抛出异常InternalH2OBackend。

我会问如何在 IDE 中运行的 spark 本地模式下运行这个示例

4

0 回答 0