2

我正在尝试运行随 HERE Open Location Platform SDK 提供的批处理示例。

我的 maven exec 看起来像这样：

mvn exec:java -Dexec.cleanupDaemonThreads=false \
  -Dexec.mainClass=com.here.platform.examples.location.batch.Main \
  -Dpipeline-config.file=config/pipeline-config.conf \
  -Dpipeline-job.file=config/pipeline-job.conf

我收到了以下异常：

java.lang.RuntimeException: Cannot setup Spark and the Driver
    at com.here.platform.data.processing.driver.DriverRunner$class.runJob (DriverRunner.scala:115)
    at com.here.platform.examples.location.batch.Main$.runJob (Main.scala:24)
    at com.here.platform.data.processing.driver.CmdLineRunner$class.run (CmdLineRunner.scala:42)
    at com.here.platform.examples.location.batch.Main$.run (Main.scala:24)
    at com.here.platform.data.processing.driver.CmdLineRunner$class.main (CmdLineRunner.scala:59)
    at com.here.platform.examples.location.batch.Main$.main (Main.scala:24)
    at com.here.platform.examples.location.batch.Main.main (Main.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
    at java.lang.Thread.run (Thread.java:748)
Caused by: org.apache.spark.SparkException: A master URL must be set in your configuration
    at org.apache.spark.SparkContext.<init> (SparkContext.scala:379)
    at com.here.platform.data.processing.driver.DriverRunner$class.com$here$platform$data$processing$driver$DriverRunner$$newSparkContext (DriverRunner.scala:176)
    at com.here.platform.data.processing.driver.DriverRunner$$anonfun$2$$anonfun$3.apply (DriverRunner.scala:106)
    at com.here.platform.data.processing.driver.DriverRunner$$anonfun$2$$anonfun$3.apply (DriverRunner.scala:106)
    at scala.Option.getOrElse (Option.scala:121)
    at com.here.platform.data.processing.driver.DriverRunner$$anonfun$2.apply (DriverRunner.scala:106)
    at com.here.platform.data.processing.driver.DriverRunner$$anonfun$2.apply (DriverRunner.scala:105)
    at scala.util.Try$.apply (Try.scala:192)
    at com.here.platform.data.processing.driver.DriverRunner$class.runJob (DriverRunner.scala:105)
    at com.here.platform.examples.location.batch.Main$.runJob (Main.scala:24)
    at com.here.platform.data.processing.driver.CmdLineRunner$class.run (CmdLineRunner.scala:42)
    at com.here.platform.examples.location.batch.Main$.run (Main.scala:24)
    at com.here.platform.data.processing.driver.CmdLineRunner$class.main (CmdLineRunner.scala:59)
    at com.here.platform.examples.location.batch.Main$.main (Main.scala:24)
    at com.here.platform.examples.location.batch.Main.main (Main.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
    at java.lang.Thread.run (Thread.java:748)
4

1 回答 1

2

好问题肯。

您遇到的错误是您的 Maven 构建中的执行参数中的一个简单遗漏。

将以下内容添加到您的 Maven 行：

-Dexec.args=--master local[*]

例如：

mvn exec:java -Dexec.cleanupDaemonThreads=false
-Dexec.mainClass=com.here.platform.examples.location.batch.Main
"-Dexec.args=--master local[*]"
-Dpipeline-config.file=config/pipeline-config.conf
-Dpipeline-job.file=config/pipeline-job.conf
于 2018-04-24T19:25:52.247 回答