我正在尝试运行我在 Hue 中创建的一个简单的工作流，它只调用一个不带参数的 Spark 作业，但在运行它时出现异常。它在 Google DataProc 集群上运行，并使用 Oozie 4.2.0 和 Hue 3.9 作为 UI 并运行它。
我已确保 mapreduce.framework.name 用于纱线。到目前为止，我在网上找到了一些解决方案，但所有这些都围绕工作流 XML 的问题。我是手工重建的，它也在做同样的事情。
这是我的工作流程的代码：
<workflow-app name="Drug_File_Price_Generation" xmlns="uri:oozie:workflow:0.5">
<start to="spark-22ca"/>
<kill name="Kill">
<message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<action name="spark-22ca">
<spark xmlns="uri:oozie:spark-action:0.1">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<master>local[*]</master>
<mode>client</mode>
<name>MySpark</name>
<class>com.company.drug.GenerateDrugPriceFile</class>
<jar>gs://customers/jar/DrugFileImport-assembly-1.0.jar</jar>
</spark>
<ok to="End"/>
<error to="Kill"/>
</action>
<end name="End"/>
这是我收到的日志的输出：
2016-03-22 19:27:27,927 INFO ActionStartXCommand:520 - SERVER[cluster-1-m.c.server-name.internal] USER[sayle.matthews] GROUP[-] TOKEN[] APP[Drug_File_Price_Generation] JOB[0000011-160322181838855-oozie-oozi-W] ACTION[0000011-160322181838855-oozie-oozi-W@:start:] Start action [0000011-160322181838855-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-03-22 19:27:27,930 INFO ActionStartXCommand:520 - SERVER[cluster-1-m.c.server-name.internal] USER[sayle.matthews] GROUP[-] TOKEN[] APP[Drug_File_Price_Generation] JOB[0000011-160322181838855-oozie-oozi-W] ACTION[0000011-160322181838855-oozie-oozi-W@:start:] [***0000011-160322181838855-oozie-oozi-W@:start:***]Action status=DONE
2016-03-22 19:27:27,930 INFO ActionStartXCommand:520 - SERVER[cluster-1-m.c.server-name.internal] USER[sayle.matthews] GROUP[-] TOKEN[] APP[Drug_File_Price_Generation] JOB[0000011-160322181838855-oozie-oozi-W] ACTION[0000011-160322181838855-oozie-oozi-W@:start:] [***0000011-160322181838855-oozie-oozi-W@:start:***]Action updated in DB!
2016-03-22 19:27:27,963 INFO WorkflowNotificationXCommand:520 - SERVER[cluster-1-m.c.server-name.internal] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000011-160322181838855-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000011-160322181838855-oozie-oozi-W
2016-03-22 19:27:27,963 INFO WorkflowNotificationXCommand:520 - SERVER[cluster-1-m.c.server-name.internal] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000011-160322181838855-oozie-oozi-W] ACTION[0000011-160322181838855-oozie-oozi-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0000011-160322181838855-oozie-oozi-W@:start:
2016-03-22 19:27:27,976 INFO ActionStartXCommand:520 - SERVER[cluster-1-m.c.server-name.internal] USER[sayle.matthews] GROUP[-] TOKEN[] APP[Drug_File_Price_Generation] JOB[0000011-160322181838855-oozie-oozi-W] ACTION[0000011-160322181838855-oozie-oozi-W@spark-22ca] Start action [0000011-160322181838855-oozie-oozi-W@spark-22ca] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-03-22 19:27:32,670 WARN ActionStartXCommand:523 - SERVER[cluster-1-m.c.server-name.internal] USER[sayle.matthews] GROUP[-] TOKEN[] APP[Drug_File_Price_Generation] JOB[0000011-160322181838855-oozie-oozi-W] ACTION[0000011-160322181838855-oozie-oozi-W@spark-22ca] Error starting action [spark-22ca]. ErrorType [TRANSIENT], ErrorCode [JA009], Message [JA009: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.]
org.apache.oozie.action.ActionExecutorException: JA009: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.
at org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:456)
at org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:436)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1132)
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1286)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:250)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:64)
at org.apache.oozie.command.XCommand.call(XCommand.java:286)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:321)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:250)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:475)
at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:454)
at org.apache.oozie.service.HadoopAccessorService$3.run(HadoopAccessorService.java:437)
at org.apache.oozie.service.HadoopAccessorService$3.run(HadoopAccessorService.java:435)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:435)
at org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:1329)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1080)
... 10 more
编辑：这里要求的是我的 mapred-site.xml 文件：
<?xml version="1.0" ?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>mapreduce.job.maps</name>
<value>20</value>
<description>
The default number of map tasks per job. Ignored when mapred.job.tracker
is &amp;amp;quot;local&amp;amp;quot;.
</description>
</property>
<property>
<name>mapreduce.map.memory.mb</name>
<value>3072</value>
<source>programatically</source>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>cluster-1-m:10020</value>
<description>MapReduce JobHistory Server IPC host:port</description>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>cluster-1-m:19888</value>
<description>MapReduce JobHistory Server Web UI host:port</description>
</property>
<property>
<name>mapreduce.reduce.memory.mb</name>
<value>6144</value>
<source>programatically</source>
</property>
<property>
<name>yarn.app.mapreduce.am.command-opts</name>
<value>-Xmx4915m</value>
<source>programatically</source>
</property>
<property>
<name>mapreduce.tasktracker.map.tasks.maximum</name>
<value>4</value>
<description>
Property from MapReduce version 1 still used for TeraGen sharding.
</description>
</property>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.input.fileinputformat.list-status.num-threads</name>
<value>20</value>
<description>
The number of threads to use to list and fetch block locations for the
specified input paths. Note: multiple threads should not be used if a
custom non thread-safe path filter is used. Setting a larger value than
the default of 1 can significantly improve job startup overhead,
especially if using GCS as input with multi-level directories, such
as in partitioned Hive tables.
</description>
</property>
<property>
<name>mapreduce.reduce.java.opts</name>
<value>-Xmx4915m</value>
<source>programatically</source>
</property>
<property>
<name>yarn.app.mapreduce.am.resource.cpu-vcores</name>
<value>2</value>
<source>programatically</source>
</property>
<property>
<name>mapreduce.reduce.cpu.vcores</name>
<value>2</value>
<source>programatically</source>
</property>
<property>
<name>mapreduce.map.cpu.vcores</name>
<value>1</value>
<source>programatically</source>
</property>
<property>
<name>yarn.app.mapreduce.am.resource.mb</name>
<value>6144</value>
<source>programatically</source>
</property>
<property>
<name>mapreduce.job.reduces</name>
<value>8</value>
<description>
The default number of reduce tasks per job. Typically set to 99%of the
cluster's reduce capacity, so that if a node fails the reduces can still
be executed in a single wave. Ignored when mapred.job.tracker is
&amp;amp;quot;local&amp;amp;quot;.
</description>
</property>
<property>
<name>mapreduce.map.java.opts</name>
<value>-Xmx2457m</value>
<source>programatically</source>
</property>
<property>
<name>mapreduce.job.working.dir</name>
<value>/user/${user.name}</value>
<description>
The FileSystem working directory to use for relative paths.
</description>
</property>
<property>
<name>mapreduce.fileoutputcommitter.algorithm.version</name>
<value>2</value>
<description>
Updated file output committer algorithm in Hadoop 2.7+. Significantly
improves commitJob times when using the Google Cloud Storage connector.
See https://issues.apache.org/jira/browse/MAPEDUCE-4815 for more details.
</description>
</property>
<property>
<name>mapred.local.dir</name>
<value>/hadoop/mapred/local</value>
<description>
Directories on the local machine in which to store mapreduce temp files.
</description>
</property>
<property>
<name>mapred.bq.project.id</name>
<value>instant-bonbon-111304</value>
<description>
Google Cloud Project ID to use for BigQuery operations.
</description>
</property>
<property>
<name>mapred.bq.output.buffer.size</name>
<value>67108864</value>
<description>
The size in bytes of the output buffer to use when writing to BigQuery.
</description>
</property>
<property>
<name>mapred.bq.gcs.bucket</name>
<value>dataproc-efab4f81-9a3a-4e62-ae58-db967541284c-us</value>
<description>
The GCS bucket holding temporary BigQuery data for the input connector.
</description>
</property>
</configuration>
编辑 2：无法让 Hue 导出 job.properties 文件，但这是我在命令行上测试时生成的文件。我相信生成的不会有太大的不同。
nameNode=hdfs://localhost:8020
jobTracker=localhost:8050
master=local[*]
oozie.wf.application.path=/home/sayle_matthews/workflow
oozie.use.system.libpath=true