1

由于在运行我的 celery 任务时出现高并发写入问题（这些在 sqlite3 上运行良好），我最近被迫将数据库从 sqlite3 切换到 mysql。现在我已经配置了 mysql 数据库，它会抛出一个

C:\Sites\deltafy-massive-feed-backend>celery -A backend worker -l info

-------------- celery@Robert-PC v3.1.11 (Cipater)
---- **** -----
--- * ***  * -- Windows-7-6.1.7601-SP1
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         backend:0x2d69b10
- ** ---------- .> transport:   django://localhost//
- ** ---------- .> results:     djcelery.backends.cache:CacheBackend
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ----
--- ***** ----- [queues]
--------------- .> celery           exchange=celery(direct) key=celery


[tasks]
  . backend.celery.debug_task
  . private_api.tasks.gilt_script
  . private_api.tasks.nordstrom_script
  . private_api.tasks.zappos_script

[2014-06-14 01:37:13,625: ERROR/MainProcess] Unrecoverable error: WindowsError(123,             'The filename, directory name, or volume label syntax is incorrect')
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\celery\worker\__init__.py", line 206, in start
    self.blueprint.start(self)
  File "C:\Python27\lib\site-packages\celery\bootsteps.py", line 123, in start
    step.start(parent)
  File "C:\Python27\lib\site-packages\celery\bootsteps.py", line 373, in start
    return self.obj.start()
  File "C:\Python27\lib\site-packages\celery\concurrency\base.py", line 131, in start
    self.on_start()
  File "C:\Python27\lib\site-packages\celery\concurrency\prefork.py", line 117, in on_start
    **self.options)
  File "C:\Python27\lib\site-packages\billiard\pool.py", line 925, in __init__
    self._setup_queues()
  File "C:\Python27\lib\site-packages\billiard\pool.py", line 1267, in _setup_queues
    self._inqueue = SimpleQueue()
  File "C:\Python27\lib\site-packages\billiard\queues.py", line 369, in __init__
    self._reader, self._writer = Pipe(duplex=False)
  File "C:\Python27\lib\site-packages\billiard\__init__.py", line 97, in Pipe
    return Pipe(duplex, rnonblock, wnonblock)
  File "C:\Python27\lib\site-packages\billiard\py2\connection.py", line 233, in Pipe
    1, obsize, ibsize, win32.NMPWAIT_WAIT_FOREVER, win32.NULL
WindowsError: [Error 123] The filename, directory name, or volume label syntax is     incorrect`

另外，我有一种预感，地址是问题所在并且包含无效字符，所以我进入了 connection.py 并打印了它。地址是：

c:\\.\pipe\pyc-5532-0-7ix5od

非常感谢有关此操作的任何想法或问题的任何解决方案！即使从我的搜索时间来看，我也没有任何线索。

4

1 回答 1

2

这个问题是 billiard 中的一个 bug，它是 Celery 使用的多处理模块分支。这是该问题的链接。

您可以通过运行来确认这一点，python -c "from billiard.connection import Pipe; p = Pipe(duplex=False)"这应该会产生相同的错误。

Cogniva刚刚将其修复到他们的台球叉上。在写这篇文章的时候，这个修复已经提交了，虽然没有合并到celery/billiard中。此问题将在台球 3.4.0.0 版本中得到解决。

要安装修复程序：

$ pip uninstall billiard
$ git clone git@github.com:cogniva/billiard.git
$ cd billiard 
$ git checkout bb6169ff1bdd8bc7c5520f7c47085063bc51d4c7
$ python setup.py install

要确认此修复程序有效，请运行python -c "from billiard.connection import Pipe; p = Pipe(duplex=False)"。这次您应该不会看到错误消息。

于 2014-06-19T19:50:10.143 回答