1

以下是我用来运行作业的所有 oozie 文件。我在 hdfs /test/jar 上创建了文件夹，并放置了 workflow.xml 和 coordinator.xml 文件。

属性文件

nameNode=hdfs://host:8020
jobTracker=host:8050
queueName=default
oozie.use.system.lib.path=trueoozie.coord.application.path=${nameNode}/test/jar/coordinator.xml
oozie.action.sharelib.for.spark=spark2
start=2019-05-22T07:37Z
end=2019-05-22T07:40Z
freq=*/1 * * * *
zone=UTC
user.name=oozie
oozie.action.sharelib.for.spark.exclusion=oozie/jackson
#oozie.libpath=${nameNode}/user/oozie/share/lib

协调员文件

<coordinator-app xmlns = "uri:oozie:coordinator:0.5" name = "test" frequency = "${freq}" start = "${start}" end = "${end}" timezone = "${zone}">
   <controls>
      <timeout>1</timeout>
   </controls>
   <action>
      <workflow>
         <app-path>${nameNode}/test/jar/workflow.xml</app-path>
      </workflow>
   </action>
</coordinator-app>

工作流文件

<workflow-app name="sample-wf" xmlns="uri:oozie:workflow:0.5">
 <start to="test" />
    <action name="test">
        <spark xmlns="uri:oozie:spark-action:0.2">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <master>yarn</master>
            <mode>cluster</mode>
            <name>Spark Example</name>
            <class>com.spark.excel.mysql.executor.Executor</class>
            <jar>${nameNode}/test/jar/com.spark.excel.mysql-0.1.jar</jar>
            <spark-opts>--executor-memory 2G --num-executors 2</spark-opts>
        </spark>
        <ok to="end"/>
        <error to="fail"/>
    </action>
 <kill name="fail">
            <message>Workflow failed, error message [${wf:errorMessage(wf:lastErrorNode())}]</message>
 </kill>
  <end name="end" />
</workflow-app>

我也设置了 sharelib 路径。Oozie 也通过 shareliblist 显示 spark2 并在 spark2.Ozzie 作业中添加了 oozie-sharelib-spark.jar 文件并正在提交和运行，但是当它尝试执行 spark 作业时出现错误。

4

0 回答 0