3

我正在尝试使用最新的 Giraph 代码和 Hadoop 2.5.2 运行 SimpleShortestPathsComputation 示例。我的命令行如下所示:

hadoop jar /home/prhodes/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.5.2-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/prhodes/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/prhodes/giraph_output/shortestpaths -w 4

这项工作似乎可以开始了。但是随后它开始输出这些类型的消息,并且这只会(似乎)永远持续下去,直到您 ctrl+c 它。

15/03/11 02:54:31 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
305.43 secs 15/03/11 02:54:31 INFO yarn.GiraphYarnClient: appattempt_1426041786848_0002_000001, State: ACCEPTED, Containers
used: 1 15/03/11 02:54:35 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
309.44 secs 15/03/11 02:54:35 INFO yarn.GiraphYarnClient: appattempt_1426041786848_0002_000001, State: ACCEPTED, Containers
used: 1 15/03/11 02:54:39 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
313.45 secs 15/03/11 02:54:39 INFO yarn.GiraphYarnClient: appattempt_1426041786848_0002_000001, State: ACCEPTED, Containers
used: 1 15/03/11 02:54:43 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
317.45 secs 15/03/11 02:54:43 INFO yarn.GiraphYarnClient: appattempt_1426041786848_0002_000001, State: ACCEPTED, Containers
used: 1 ^C15/03/11 02:54:47 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
321.46 secs 15/03/11 02:54:47 INFO yarn.GiraphYarnClient: appattempt_1426041786848_0002_000001, State: ACCEPTED, Containers
used: 1

知道这里发生了什么吗?

4

1 回答 1

0

结果证明这是一个环境问题。我试图在我的 AWS 安全组中选择要列入白名单的端口,但错过了 YARN 使用一些端口在集群上进行内部通信。我没有尝试对整个列表进行排序,而是添加了一个安全组规则,用于从 10.0.0.0/24 接受“所有流量”。这解决了这个问题。

于 2015-03-12T03:39:59.347 回答