2

当我在管道版本中运行它们时，我有一个映射器和减速器可以正常工作：

cat data.csv | ./mapper.py | sort -k1,1 | ./reducer.py

我使用了elastic mapreducer 向导，加载了输入、输出、引导程序等。引导程序是成功的，但是我在执行过程中仍然出现错误。

这是我在第 1 步的 stderr 中遇到的错误...

+ /etc/init.d/hadoop-state-pusher-control stop
+ PID_FILE=/mnt/var/run/hadoop-state-pusher/hadoop-state-pusher.pid
+ LOG_FILE=/mnt/var/log/hadoop-state-pusher/hadoop-state-pusher.out
+ SVC_FILE=/mnt/var/lib/hadoop-state-pusher/run-hadoop-state-pusher
+ case $1 in
+ stop
+ echo 0
/etc/init.d/hadoop-state-pusher-control: line 35: /mnt/var/lib/hadoop-state-pusher/run-hadoop-state-pusher: No such file or directory
+ /etc/init.d/hadoop-state-pusher-control start
+ PID_FILE=/mnt/var/run/hadoop-state-pusher/hadoop-state-pusher.pid
+ LOG_FILE=/mnt/var/log/hadoop-state-pusher/hadoop-state-pusher.out
+ SVC_FILE=/mnt/var/lib/hadoop-state-pusher/run-hadoop-state-pusher
+ case $1 in
+ start
++ dirname /mnt/var/lib/hadoop-state-pusher/run-hadoop-state-pusher
+ sudo -u hadoop mkdir -p /mnt/var/lib/hadoop-state-pusher
+ echo 1
++ dirname /mnt/var/run/hadoop-state-pusher/hadoop-state-pusher.pid
+ sudo -u hadoop mkdir -p /mnt/var/run/hadoop-state-pusher
++ dirname /mnt/var/log/hadoop-state-pusher/hadoop-state-pusher.out
+ sudo -u hadoop mkdir -p /mnt/var/log/hadoop-state-pusher
+ disown %1
+ sleep 5
+ sudo -u hadoop /usr/bin/hadoop-state-pusher -server --pidfile /mnt/var/run/hadoop-state-pusher/hadoop-state-pusher.pid
+ exit 0
Command exiting with ret '0'

这是神秘的。这到底是什么意思？

安装东西似乎有问题？其他哪些日志文件可能会提供一些信息，我应该在哪里查找？

我尝试了一个我在这里找到的解决方案，只是让实例更大，但这不起作用，同样的错误消息。

4

1 回答 1

0

我正在查看错误的日志文件。有一个不同的（有 6 个？）实际上给了我一些有用的 python 调试信息。事实证明，我使用了string interpolation.format("of this kind {}, not this kind with a digit {1}".vars(a,b))python < 2.7 中不支持的 a，它是默认安装在弹性 mapreduce 中使用的 EC2 映像上的。

于 2013-09-30T14:58:15.803 回答