1

我试图让最简单的 Hadoop“hello world”设置工作，但是当我运行以下命令时：

hadoop jar /usr/share/hadoop/hadoop-examples-1.0.4.jar grep input output 'dfs[a-z.]+'

我收到以下警告：

12/11/30 16:36:40 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

完整的错误跟踪如下所示：

12/11/30 16:57:18 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
12/11/30 16:57:18 WARN snappy.LoadSnappy: Snappy native library not loaded
12/11/30 16:57:18 INFO mapred.FileInputFormat: Total input paths to process : 6
12/11/30 16:57:18 INFO mapred.JobClient: Running job: job_local_0001
12/11/30 16:57:18 INFO util.ProcessTree: setsid exited with exit code 0
12/11/30 16:57:18 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7d4ecfa4
12/11/30 16:57:18 INFO mapred.MapTask: numReduceTasks: 1
12/11/30 16:57:18 INFO mapred.MapTask: io.sort.mb = 100
12/11/30 16:57:18 WARN mapred.LocalJobRunner: job_local_0001
java.lang.OutOfMemoryError: Java heap space
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:949)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:428)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
12/11/30 16:57:19 INFO mapred.JobClient:  map 0% reduce 0%
12/11/30 16:57:19 INFO mapred.JobClient: Job complete: job_local_0001
12/11/30 16:57:19 INFO mapred.JobClient: Counters: 0
12/11/30 16:57:19 INFO mapred.JobClient: Job Failed: NA
java.io.IOException: Job failed!
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1265)
    at org.apache.hadoop.examples.Grep.run(Grep.java:69)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.examples.Grep.main(Grep.java:93)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

我正在运行 Ubuntu 12.04 和 Java 版本：

java version "1.7.0"
Java(TM) SE Runtime Environment (build 1.7.0-b147)
Java HotSpot(TM) 64-Bit Server VM (build 21.0-b17, mixed mode)

有任何想法吗？

4

1 回答 1

2

警告告诉您未（正确）为 Hadoop 安装压缩编解码器。要安装Snappy压缩，请查看：http ://code.google.com/p/hadoop-snappy/

然而，一个更严重的问题是OutOfMemoryError你得到的。检查您的输入，如有必要，增加堆大小。您可能还会查看这个相关问题：
Hadoop 中的内存不足错误

于 2012-12-01T22:44:25.210 回答