2

运行 Django 1.4.10、Celery 3.1.7、Python 2.7、Kombu 3.0.12,并使用 django supervisor 将 celery worker 和 celery beat 作为守护进程运行。使用 IronMQ 作为代理。

一切正常处理几天的周期性任务,然后突然之间,工人将致命地死去:

[2014-08-04 16:52:17,647: ERROR/MainProcess] Unrecoverable error: TypeError("sequence index must be integer, not 'unicode'",)
Traceback (most recent call last):
  File "/home/kromedev/webapps/gorilla/lib/python2.7/celery-3.1.7-py2.7.egg/celery/worker  /__init__.py", line 206, in start
self.blueprint.start(self)
  File "/home/kromedev/webapps/gorilla/lib/python2.7/celery-3.1.7-py2.7.egg/celery/bootsteps.py", line 123, in start
step.start(parent)
  File "/home/kromedev/webapps/gorilla/lib/python2.7/celery-3.1.7-py2.7.egg/celery/bootsteps.py", line 373, in start
return self.obj.start()
  File "/home/kromedev/webapps/gorilla/lib/python2.7/celery-3.1.7-py2.7.egg/celery/worker/consumer.py", line 270, in start
blueprint.start(self)
  File "/home/kromedev/webapps/gorilla/lib/python2.7/celery-3.1.7-py2.7.egg/celery/bootsteps.py", line 123, in start
step.start(parent)
  File "/home/kromedev/webapps/gorilla/lib/python2.7/celery-3.1.7-py2.7.egg/celery/worker/consumer.py", line 786, in start
c.loop(*c.loop_args())
  File "/home/kromedev/webapps/gorilla/lib/python2.7/celery-3.1.7-py2.7.egg/celery/worker/loops.py", line 99, in synloop
connection.drain_events(timeout=2.0)
  File "/home/kromedev/webapps/gorilla/lib/python2.7/kombu-3.0.12-py2.7.egg/kombu/connection.py", line 279, in drain_events
return self.transport.drain_events(self.connection, **kwargs)
  File "/home/kromedev/webapps/gorilla/lib/python2.7/kombu-3.0.12-py2.7.egg/kombu/transport/virtual/__init__.py", line 844, in drain_events
self._callbacks[queue](message)
  File "/home/kromedev/webapps/gorilla/lib/python2.7/kombu-3.0.12-py2.7.egg/kombu/transport/virtual/__init__.py", line 529, in _callback
message = self.Message(self, raw_message)
  File "/home/kromedev/webapps/gorilla/lib/python2.7/kombu-3.0.12-py2.7.egg/kombu/transport/virtual/__init__.py", line 242, in __init__
properties = payload['properties']
TypeError: sequence index must be integer, not 'unicode'

查看时间戳,似乎总是在任务全部完成后崩溃。当我有我的 supervisor.conf 文件来自动重启时,它会进入一个循环,试图重启 worker 并立即失败。但是,如果我用主管关闭所有进程并重新启动,它会再次运行几天。

主管.conf:

[program:celeryd]
command={{ PYTHON }} {{ PROJECT_DIR }}/manage.py celery worker -l info
numprocs=1
stdout_logfile={{ PROJECT_DIR }}/worker.log
stderr_logfile={{ PROJECT_DIR }}/worker.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs=600
killasgroup=true
priority=998

[program:celerybeat]
command={{ PYTHON }} {{ PROJECT_DIR }}/manage.py celery beat  -l INFO
numprocs=1
stdout_logfile={{ PROJECT_DIR }}/beat.log
stderr_logfile={{ PROJECT_DIR }}/beat.log
autostart=true
autorestart=true
startsecs=10
priority=999 

甚至不知道从哪里开始调试这个 - 把它扔在那里,以防其他人遇到同样的问题并且可以对此有所了解。

更新:我切换回可靠的 Erlang/Rabbit,问题完全消失了。所以它是 IronMQ。我真的无法从他们那里得到任何帮助,所以他们失去了一个客户!

4

0 回答 0