所以我正在运行windows 10,python3.9.x,使用aws sqs作为代理,刚刚完成了一些文件的更新:
设置.py
###
### For celery tasks!
###
from kombu.utils.url import safequote
import urllib.parse
AWS_ACCESS_KEY_ID = 'my aws_access_key_id for a user'
AWS_SECRET_ACCESS_KEY = 'my aws_secret_access_key for a user'
BROKER_URL = 'sqs://%s:%s@' % (urllib.parse.quote(AWS_ACCESS_KEY_ID, safe=''), urllib.parse.quote(AWS_SECRET_ACCESS_KEY, safe=''))
BROKER_TRANSPORT = 'sqs'
BROKER_TRANSPORT_OPTIONS = {
'canves-celery-queue': {
'access_key_id': safequote(AWS_ACCESS_KEY_ID),
'secret_access_key': safequote(AWS_SECRET_ACCESS_KEY),
'region': 'us-east-1'
}
}
CELERY_DEFAULT_QUEUE = 'celery<-project-queue>'
CELERY_QUEUES = {
CELERY_DEFAULT_QUEUE: {
'exchange': CELERY_DEFAULT_QUEUE,
'binding_key': CELERY_DEFAULT_QUEUE,
}
}
###
### End celery tasks
###
celery_tasks.py(在本教程中称为 celery.py - 重命名是因为显然这会导致其他一些程序员出现一些错误):
from __future__ import absolute_import
import os
from celery import Celery
# Set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', '<project>.settings')
# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
# should have a `CELERY_` prefix.
app = Celery('<project>', include=['<project>.tasks'])
app.config_from_object('django.conf:settings')
# Load task modules from all registered Django apps.
app.autodiscover_tasks()
@app.task(bind=True)
def debug_task(self):
print("debug_task was fired!")
print(f'Request: {self.request!r}')
if __name__ == '__main__':
app.start()
tasks.py(这与 settings.py 在同一目录中 - 也在 celery_tasks.py 中引用)
from __future__ import absolute_import
from .celery_tasks import app
import time
@app.task(ignore_result=True)
def sleep(x, y):
print("Sleeping for: " + str(x + y))
time.sleep(x + y)
print("Slept for: " + str(x + y))
当我去运行worker时(确保你和manage.py在同一个目录中),它抛出了这个错误:
from kombu.async.timer import Entry, Timer as Schedule, to_timestamp, logger
为了修复它,我按照 gogaz 的回答跑了
pip uninstall django-celery
pip uninstall celery && pip install celery
这将我推到了最新版本的 celery, 4.3 ... celery 4+ is not supported on windows as per this SO question ( Celery raises ValueError: not enough values to unpack ), 方便地有这个答案 (由 Samuel Chen 发布):
适用于 celery 4.2+,python3,windows 10
pip install gevent
celery -A <project> worker -l info -P gevent
适用于 celery 4.1+、python3、windows 10
pip install eventlet
celery -A <project> worker -l info -P eventlet
我得到的唯一另一个错误是 django 的调试器正在打开,这显然会导致内存泄漏......
问题(至少对我来说)是我不能使用 Prefork 池,这意味着我不能使用 app.control.revoke() 来终止任务。