0

一段时间以来，我一直在我的 Mac 上使用 Hadoop (CDH 4.5)，并定期运行 map reduce 作业。我最近按照以下说明安装了 oozie（再次是 CDH4.5）：http: //archive.cloudera.com/cdh4/cdh/4/oozie-3.3.2-cdh4.5.0/DG_QuickStart.html，并尝试运行示例提供的程序。但是，它总是失败并出现以下错误。看起来工作流程根本没有运行。作业信息中的控制台 URL 字段也是空的。

有人可以帮忙吗？

Oozie Job 日志的相关片段如下。

2014-06-10 17:27:18,414  INFO ActionStartXCommand:539 - USER[userXXX] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-140610172702069-oozie-usrX-W] ACTION[0000000-140610172702069-oozie-usrX-W@:start:] Start action [0000000-140610172702069-oozie-usrX-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-06-10 17:27:18,417  WARN ActionStartXCommand:542 - USER[userXXX] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-140610172702069-oozie-usrX-W] ACTION[0000000-140610172702069-oozie-usrX-W@:start:] [***0000000-140610172702069-oozie-usrX-W@:start:***]Action status=DONE
2014-06-10 17:27:18,417  WARN ActionStartXCommand:542 - USER[userXXX] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-140610172702069-oozie-usrX-W] ACTION[0000000-140610172702069-oozie-usrX-W@:start:] [***0000000-140610172702069-oozie-usrX-W@:start:***]Action updated in DB!
2014-06-10 17:27:18,576  INFO ActionStartXCommand:539 - USER[userXXX] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-140610172702069-oozie-usrX-W] ACTION[0000000-140610172702069-oozie-usrX-W@mr-node] Start action [0000000-140610172702069-oozie-usrX-W@mr-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-06-10 17:27:19,188  WARN MapReduceActionExecutor:542 - USER[userXXX] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-140610172702069-oozie-usrX-W] ACTION[0000000-140610172702069-oozie-usrX-W@mr-node] credentials is null for the action
2014-06-10 17:27:19,423  WARN ActionStartXCommand:542 - USER[userXXX] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-140610172702069-oozie-usrX-W] ACTION[0000000-140610172702069-oozie-usrX-W@mr-node] Error starting action [mr-node]. ErrorType [TRANSIENT], ErrorCode [JA009], Message [JA009: Unknown rpc kind RPC_WRITABLE]
org.apache.oozie.action.ActionExecutorException: JA009: Unknown rpc kind RPC_WRITABLE
at org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:418)
at org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:773)
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:927)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:211)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:59)
at org.apache.oozie.command.XCommand.call(XCommand.java:277)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:326)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:255)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.hadoop.ipc.RemoteException(java.io.IOException): Unknown rpc kind RPC_WRITABLE
at org.apache.hadoop.ipc.Client.call(Client.java:1238)
at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:225)
at org.apache.hadoop.mapred.$Proxy30.getDelegationToken(Unknown Source)
at org.apache.hadoop.mapred.JobClient.getDelegationToken(JobClient.java:2125)
at org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:372)
at org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:970)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:723)
... 10 more

2014-06-10 17:27:19,426  INFO ActionStartXCommand:539 - USER[userXXX] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-140610172702069-oozie-usrX-W] ACTION[0000000-140610172702069-oozie-usrX-W@mr-node] Next Retry, Attempt Number [1] in [60,000] milliseconds
2014-06-10 17:28:19,468  INFO ActionStartXCommand:539 - USER[userXXX] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-140610172702069-oozie-usrX-W] ACTION[0000000-140610172702069-oozie-usrX-W@mr-node] Start action [0000000-140610172702069-oozie-usrX-W@mr-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
4

1 回答 1

0

您是否使用 MapReduce 或 YARN 设置了 hadoop？

Configuring which Hadoop Version to Use
The Oozie client does not interact directly with Hadoop MapReduce, and so it does not require any MapReduce configuration.

The Oozie server can work with either MRv1 or YARN. It cannot work with both simultaneously.

The MapReduce version Oozie server works with is determined by the CATALINA_BASE variable in the /etc/oozie/conf/oozie-env.sh file. By default, CATALINA_BASE is set to /usr/lib/oozie/oozie-server-0.20. This setting configures the Oozie server to work with MRv1.

To configure the Oozie server to work with YARN instead, set CATALINA_BASE to /usr/lib/oozie/oozie-server.

由于您有 cdh4 发行版，请按照此链接http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_topic_17.html安装 oozie

如果你有 YARN，在你的 job.properties 中更改 jobtracker 的端口

nameNode=hdfs://localhost:8020 
jobTracker=localhost:8021

代替端口 8021 (jobTracker)，使用资源管理器的端口：8032

jobTracker=localhost:8032

https://groups.google.com/a/cloudera.org/forum/#!topic/cdh-user/MlsqeUdMpgA

于 2014-06-12T11:12:56.240 回答