2

我在 Amazon EC2 实例中设置了单节点 hadoop。按照这个和这个，我可以第一次运行示例程序。但要让它第二次运行，我必须在停止发出 stop-all.sh 后删除 S3 和本地 tmp 目录中的所有目录和文件。我只运行 mapred（tasktracker 和 jobtracker）。尝试第二次重新运行该示例，我收到错误消息。

        hduser@ip-10-252-196-143:~$ hadoop jar ./hadoop/hadoop-examples-1.2.1.jar  wordcount input output2
13/09/20 09:43:06 ERROR security.UserGroupInformation: PriviledgedActionException as:hduser cause:java.io.IOException: The ownership on the staging directory s3://vikesh-hadoop-bucket/home/hduser/tmp/mapred/staging/hduser/.staging is not as expected. It is owned by . The directory must be owned by the submitter hduser or by hduser
java.io.IOException: The ownership on the staging directory s3://vikesh-hadoop-bucket/home/hduser/tmp/mapred/staging/hduser/.staging is not as expected. It is owned by . The directory must be owned by the submitter hduser or by hduser
        at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:113)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:942)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:550)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:580)
        at org.apache.hadoop.examples.WordCount.main(WordCount.java:82)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:160)

似乎这是相关的，但我相信正在使用 Cloudera 的发行版。

为什么会发生这种情况，我能做些什么来解决这个问题？我在 64 位 VM 上使用带有 OpenJDK 7 的 hadoop-1.2.1。

谢谢

4

1 回答 1

1

我们面临同样的问题。我们只需删除每个 MapReduce 之间的 s3://{bucket-name}/home/{user}/tmp/mapred/staging/{user}/.staging ... 但是如果有人有更好的解决方案...

于 2013-09-28T14:27:14.110 回答