运行我的 Hadoop 程序时,它突然无缘无故地失败了。我已附加输出消息。这似乎是一个IO问题。它告诉第 525 行一定有问题,这是我运行作业的行(第 525 行:)RunningJob runjob = JobClient.runJob(conf);
。从错误消息中可以看出,它没有更多地说明错误行。
我运行了几次,当它完成 86% 时它总是给我这个错误。有人知道失败的原因吗?
13/10/14 10:25:11 INFO mapred.JobClient: map 79% reduce 6%
13/10/14 10:25:16 INFO mapred.JobClient: map 80% reduce 6%
13/10/14 10:25:43 INFO mapred.JobClient: map 81% reduce 6%
13/10/14 10:26:13 INFO mapred.JobClient: map 82% reduce 6%
13/10/14 10:26:41 INFO mapred.JobClient: map 83% reduce 6%
13/10/14 10:26:56 INFO mapred.JobClient: map 83% reduce 8%
13/10/14 10:27:09 INFO mapred.JobClient: map 84% reduce 8%
13/10/14 10:27:44 INFO mapred.JobClient: map 85% reduce 8%
13/10/14 10:28:29 INFO mapred.JobClient: map 86% reduce 8%
13/10/14 10:29:13 INFO mapred.JobClient: Job complete: job_201310101346_0102
13/10/14 10:29:13 INFO mapred.JobClient: Counters: 31
13/10/14 10:29:13 INFO mapred.JobClient: File System Counters
13/10/14 10:29:13 INFO mapred.JobClient: FILE: Number of bytes read=316172729
13/10/14 10:29:13 INFO mapred.JobClient: FILE: Number of bytes written=501348856
13/10/14 10:29:13 INFO mapred.JobClient: FILE: Number of read operations=0
13/10/14 10:29:13 INFO mapred.JobClient: FILE: Number of large read operations=0
13/10/14 10:29:13 INFO mapred.JobClient: FILE: Number of write operations=0
13/10/14 10:29:13 INFO mapred.JobClient: HDFS: Number of bytes read=2144535743
13/10/14 10:29:13 INFO mapred.JobClient: HDFS: Number of bytes written=0
13/10/14 10:29:13 INFO mapred.JobClient: HDFS: Number of read operations=128
13/10/14 10:29:13 INFO mapred.JobClient: HDFS: Number of large read operations=0
13/10/14 10:29:13 INFO mapred.JobClient: HDFS: Number of write operations=0
13/10/14 10:29:13 INFO mapred.JobClient: Job Counters
13/10/14 10:29:13 INFO mapred.JobClient: Failed map tasks=1
13/10/14 10:29:13 INFO mapred.JobClient: Launched map tasks=188
13/10/14 10:29:13 INFO mapred.JobClient: Launched reduce tasks=1
13/10/14 10:29:13 INFO mapred.JobClient: Data-local map tasks=125
13/10/14 10:29:13 INFO mapred.JobClient: Rack-local map tasks=63
13/10/14 10:29:13 INFO mapred.JobClient: Total time spent by all maps in occupied slots (ms)=211239444
13/10/14 10:29:13 INFO mapred.JobClient: Total time spent by all reduces in occupied slots (ms)=406123
13/10/14 10:29:13 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
13/10/14 10:29:13 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
13/10/14 10:29:13 INFO mapred.JobClient: Map-Reduce Framework
13/10/14 10:29:13 INFO mapred.JobClient: Map input records=45927922
13/10/14 10:29:13 INFO mapred.JobClient: Map output records=45927922
13/10/14 10:29:13 INFO mapred.JobClient: Map output bytes=11573153427
13/10/14 10:29:13 INFO mapred.JobClient: Input split bytes=8000
13/10/14 10:29:13 INFO mapred.JobClient: Combine input records=48701007
13/10/14 10:29:13 INFO mapred.JobClient: Combine output records=3833295
13/10/14 10:29:13 INFO mapred.JobClient: Spilled Records=3833295
13/10/14 10:29:13 INFO mapred.JobClient: CPU time spent (ms)=86130400
13/10/14 10:29:13 INFO mapred.JobClient: Physical memory (bytes) snapshot=13896142848
13/10/14 10:29:13 INFO mapred.JobClient: Virtual memory (bytes) snapshot=25477984256
13/10/14 10:29:13 INFO mapred.JobClient: Total committed heap usage (bytes)=9854910464
13/10/14 10:29:13 INFO mapred.JobClient: org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter
13/10/14 10:29:13 INFO mapred.JobClient: BYTES_READ=2144450681
13/10/14 10:29:13 INFO mapred.JobClient: Job Failed: NA
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1373)
at Mondrian3.main(Mondrian3.java:525)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)