我正在运行 hadoop-examples-1.0.3.jar 文件中的打包示例之一:特别是 PiEstimator,如下所示:
hadoop jar hadoop-examples-1.0.3.jar pi 5 10
但这失败了,ClassNotFoundException:
Number of Maps = 5
Samples per Map = 10
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Starting Job
13/06/14 19:42:07 INFO mapred.JobClient: Default number of map tasks: 5
13/06/14 19:42:07 INFO mapred.JobClient: Default number of reduce tasks: 1
13/06/14 19:42:07 INFO security.ShellBasedUnixGroupsMapping: add hadoop to shell userGroupsCache
13/06/14 19:42:07 INFO mapred.JobClient: Setting group to hadoop
13/06/14 19:42:07 INFO mapred.FileInputFormat: Total input paths to process : 5
13/06/14 19:42:08 INFO mapred.JobClient: Running job: job_201306141855_0004
13/06/14 19:42:09 INFO mapred.JobClient: map 0% reduce 0%
13/06/14 19:42:17 INFO mapred.JobClient: Task Id : attempt_201306141855_0004_m_000006_0, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Task process exit with nonzero status of 1.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
attempt_201306141855_0004_m_000006_0: Exception in thread "main" java.lang.NoClassDefFoundError:
attempt_201306141855_0004_m_000006_0: Caused by: java.lang.ClassNotFoundException:
attempt_201306141855_0004_m_000006_0: at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
attempt_201306141855_0004_m_000006_0: at java.security.AccessController.doPrivileged(Native Method)
attempt_201306141855_0004_m_000006_0: at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
attempt_201306141855_0004_m_000006_0: at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
attempt_201306141855_0004_m_000006_0: at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
attempt_201306141855_0004_m_000006_0: at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
attempt_201306141855_0004_m_000006_0: Could not find the main class: . Program will exit.
13/06/14 19:42:23 INFO mapred.JobClient: Task Id : attempt_201306141855_0004_r_000002_0, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Task process exit with nonzero status of 1.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
现在我查看了job.xml文件,找到了生成的mapred.jar并从hdfs下载了它:
从 JobTracker 用户界面:
mapred.jar hdfs://10.172.86.250:9000/mnt/var/lib/hadoop/tmp/mapred/staging/hadoop/.staging/job_201306141855_0001/job.jar
下载到本地查看:
hadoop dfs -get hdfs://10.172.86.250:9000/mnt/var/lib/hadoop/tmp/mapred/staging/hadoop/.staging/job_201306141855_0001/job.jar .
现在 PiEstimator 类在那里:
jar -tvf job.jar | grep Pi
2711 Fri May 17 00:14:04 UTC 2013 org/apache/hadoop/examples/MultiFileWordCount.class
1504 Fri May 17 00:14:04 UTC 2013 org/apache/hadoop/examples/PiEstimator$HaltonSequence.class
3101 Fri May 17 00:14:04 UTC 2013 org/apache/hadoop/examples/PiEstimator$PiMapper.class
3883 Fri May 17 00:14:04 UTC 2013 org/apache/hadoop/examples/PiEstimator$PiReducer.class
7140 Fri May 17 00:14:04 UTC 2013 org/apache/hadoop/examples/PiEstimator.class
所以..提示/帮助表示赞赏。顺便说一句,我已经在数十个集群中运行了数百个 m/r 作业,所以这不完全是一个菜鸟问题。但是,我以前没有遇到过这种特殊的故障模式。