1

我有一个芹菜工人，其中包括一个在远程机器上运行木偶代理的任务，这是一个阻塞调用。我在其他可以从 eventlet 中受益的任务中也有其他 http api 流

@app.task(soft_time_limit=600)
def run_puppet_agent(hostname):
    try:
        env.host_string = self.host
        env.user = self.username
        env.password = self.password
        return sudo('puppet agent -t')
    except SoftTimeLimitExceeded:
        raise Exception('Puppet agent TIMED OUT AFTER 600 seconds')
    except Exception as e:
        #run_puppet_agent.retry(args=[hostname], countdown=20)
        LOG.info('')
        raise Exception('Puppet agent failed with error message %s' % e.message)

当我运行工人时

celery multi start 2 -A phantom.celery.manage -P eventlet -c 15 --loglevel=INFO

它给了我一个异常跟踪，如下所示：

Traceback (most recent call last):
File "/home/uruddarraju/Phantom/phantom/tasks/fabric_tasks.py", line 19, in run_puppet_agent
return sudo(command, quiet=False)
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/fabric/network.py", line 639, in host_prompting_wrapper
return func(*args, **kwargs)
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/fabric/operations.py", line 1095, in sudo
stderr=stderr, timeout=timeout, shell_escape=shell_escape,
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/fabric/operations.py", line 911, in _run_command
stderr=stderr, timeout=timeout)
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/fabric/operations.py", line 795, in _execute
worker.raise_if_needed()
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/fabric/thread_handling.py", line 12, in wrapper
callable(*args, **kwargs)
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/fabric/io.py", line 231, in input_loop
r, w, x = select([sys.stdin], [], [], 0.0)
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/eventlet/green/select.py", line 79, in select
listeners.append(hub.add(hub.READ, k, on_read))
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/eventlet/hubs/epolls.py", line 52, in add
self.register(fileno, new=True)
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/eventlet/hubs/poll.py", line 45, in register
self.poll.register(fileno, mask)
IOError: [Errno 1] Operation not permitted

我已经阅读：为什么 select.select() 可以处理磁盘文件而不是 epoll()？并查看 eventlet 使用结构或此类阻塞调用存在问题。有没有办法告诉芹菜不要修补这个特定的任务？

4

1 回答 1

1

当我改用 gevent 时，这个错误得到了缓解。我希望它能为我带来同样的表现。

于 2014-08-10T04:17:30.620 回答