1

您好我正在尝试配置 Hadoop 1.0。按照这个博客在伪分布式模式下。

http://hadoop-tutorial.blogspot.de/2010/11/running-hadoop-in-pseudo-distributed.html?showComment=1337083501000#c615470573579885293。

但是当我运行 hadoop 发行版中给出的 pi 示例时，我得到了标题中提到的错误。有人可以帮助我并指导我如何解决这个问题。它也是一个请求，如果可能的话，请提出解决方案以及确定问题。

这是我通过运行 jps 得到的

8322 Jps
7611 SecondaryNameNode
7474 DataNode
7341 NameNode

这是完整的错误消息。

Number of Maps  = 10
Samples per Map = 100
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Starting Job
12/05/16 13:11:56 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 0 time(s).
12/05/16 13:11:57 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 1 time(s).
12/05/16 13:11:58 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 2 time(s).
12/05/16 13:11:59 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 3 time(s).
12/05/16 13:12:00 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 4 time(s).
12/05/16 13:12:01 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 5 time(s).
12/05/16 13:12:02 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 6 time(s).
12/05/16 13:12:03 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 7 time(s).
12/05/16 13:12:04 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 8 time(s).
12/05/16 13:12:05 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 9 time(s).
java.net.ConnectException: Call to localhost/127.0.0.1:8021 failed on connection exception: java.net.ConnectException: Connection refused
        at org.apache.hadoop.ipc.Client.wrapException(Client.java:1095)
        at org.apache.hadoop.ipc.Client.call(Client.java:1071)
        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
        at org.apache.hadoop.mapred.$Proxy2.getProtocolVersion(Unknown Source)
        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
        at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:480)
        at org.apache.hadoop.mapred.JobClient.init(JobClient.java:474)
        at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:457)
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1260)
        at org.apache.hadoop.examples.PiEstimator.estimate(PiEstimator.java:297)
        at org.apache.hadoop.examples.PiEstimator.run(PiEstimator.java:342)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.examples.PiEstimator.main(PiEstimator.java:351)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
        at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:656)
        at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:434)
        at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:560)
        at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:184)
        at org.apache.hadoop.ipc.Client.getConnection(Client.java:1202)
        at org.apache.hadoop.ipc.Client.call(Client.java:1046)
        ... 24 more
4

4 回答 4

3

您的jps输出显示您没有运行 JobTracker 或 TaskTracker - 因此尝试与 jobtracker 通信时出现连接问题。

检查 jobtracker ( $HADOOP_HOME/logs/*.jobtracker*.log) 的日志以查看是否有错误消息描述它未启动的原因（您是如何启动集群的 -start-all.sh或者start-dfs.sh，也许尝试运行start-mapred.sh以查看是否启动了 map reduce 服务）？

于 2012-05-16T13:42:33.670 回答
0

From the jps output looks like the TaskTracker and JobTracker are not running. Try running: 

start-mapred.sh

This would start the TaskTracker and JobTracker and try rerunning job.

于 2013-02-10T23:07:35.587 回答
0

我遇到了同样的问题，这是因为 hadoop 1.0.x 有不同需要在 hadoop_env.sh 中配置两个变量：

导出 HADOOP_HEAPSIZE=2000

额外的 Java 运行时选项。默认为空。

导出 HADOOP_OPTS=-服务器

导出 HADOOP_OPTS="-Djava.security.krb5.realm=OX.AC.UK -Djava.security.krb5.kdc=kdc0.ox.ac.uk:kdc1.ox.ac.uk"

添加这两个变量时，在启动守护进程期间不会看到任何错误或警告，以便您解决问题。祝你好运！

希望我的回答可以帮助其他人:)

于 2012-12-21T07:09:58.023 回答
0

看起来这是一些与连接相关的问题。你的 jobtracker 和 trasktracker 运行正常吗？

看到这个链接

编辑：

阅读权威指南中的附录 A。它具有在所有 3 种模式下设置 hadoop 服务器的完整步骤。我认为您的配置可能会产生问题，因此请使用书中规定的配置。还要检查无密码 SSH 在您的设置中是否正常工作。

于 2012-05-16T11:53:41.783 回答