0

我正在运行 hadoop-examples-1.0.3.jar 文件中的打包示例之一:特别是 PiEstimator,如下所示:

hadoop jar hadoop-examples-1.0.3.jar pi 5 10

但这失败了,ClassNotFoundException:

Number of Maps  = 5
Samples per Map = 10
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Starting Job
13/06/14 19:42:07 INFO mapred.JobClient: Default number of map tasks: 5
13/06/14 19:42:07 INFO mapred.JobClient: Default number of reduce tasks: 1
13/06/14 19:42:07 INFO security.ShellBasedUnixGroupsMapping: add hadoop to shell userGroupsCache
13/06/14 19:42:07 INFO mapred.JobClient: Setting group to hadoop
13/06/14 19:42:07 INFO mapred.FileInputFormat: Total input paths to process : 5
13/06/14 19:42:08 INFO mapred.JobClient: Running job: job_201306141855_0004
13/06/14 19:42:09 INFO mapred.JobClient:  map 0% reduce 0%
13/06/14 19:42:17 INFO mapred.JobClient: Task Id : attempt_201306141855_0004_m_000006_0, Status : FAILED
java.lang.Throwable: Child Error
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Task process exit with nonzero status of 1.
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)

attempt_201306141855_0004_m_000006_0: Exception in thread "main" java.lang.NoClassDefFoundError: 
attempt_201306141855_0004_m_000006_0: Caused by: java.lang.ClassNotFoundException: 
attempt_201306141855_0004_m_000006_0:   at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
attempt_201306141855_0004_m_000006_0:   at java.security.AccessController.doPrivileged(Native Method)
attempt_201306141855_0004_m_000006_0:   at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
attempt_201306141855_0004_m_000006_0:   at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
attempt_201306141855_0004_m_000006_0:   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
attempt_201306141855_0004_m_000006_0:   at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
attempt_201306141855_0004_m_000006_0: Could not find the main class: .  Program will exit.
13/06/14 19:42:23 INFO mapred.JobClient: Task Id : attempt_201306141855_0004_r_000002_0, Status : FAILED
java.lang.Throwable: Child Error
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Task process exit with nonzero status of 1.
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)

现在我查看了job.xml文件,找到了生成的mapred.jar并从hdfs下载了它:

从 JobTracker 用户界面:

mapred.jar  hdfs://10.172.86.250:9000/mnt/var/lib/hadoop/tmp/mapred/staging/hadoop/.staging/job_201306141855_0001/job.jar

下载到本地查看:

hadoop dfs -get hdfs://10.172.86.250:9000/mnt/var/lib/hadoop/tmp/mapred/staging/hadoop/.staging/job_201306141855_0001/job.jar .

现在 PiEstimator 类在那里:

jar -tvf job.jar | grep Pi

  2711 Fri May 17 00:14:04 UTC 2013 org/apache/hadoop/examples/MultiFileWordCount.class
  1504 Fri May 17 00:14:04 UTC 2013 org/apache/hadoop/examples/PiEstimator$HaltonSequence.class
  3101 Fri May 17 00:14:04 UTC 2013 org/apache/hadoop/examples/PiEstimator$PiMapper.class
  3883 Fri May 17 00:14:04 UTC 2013 org/apache/hadoop/examples/PiEstimator$PiReducer.class
  7140 Fri May 17 00:14:04 UTC 2013 org/apache/hadoop/examples/PiEstimator.class

所以..提示/帮助表示赞赏。顺便说一句,我已经在数十个集群中运行了数百个 m/r 作业,所以这不完全是一个菜鸟问题。但是,我以前没有遇到过这种特殊的故障模式。

4

1 回答 1

0

命令行变得乱码(-Xmx 选项)。它

于 2013-06-14T21:09:19.177 回答