我正在使用Django、Celery和RabbitMQ。我有一个发送电子邮件的简单任务。此任务有效,但速度很慢。
例如,我发送了5000 封电子邮件,所有 5000 封电子邮件都像往常一样直接发送到RabbitMQ,但是一旦进入消息代理,它就需要大约 30 分钟才能完成并清除所有任务。
如果没有 Celery,这些相同的任务只需几分钟即可处理所有 5000 个任务。
我错过了配置的东西吗?如果有人能发现我的速度问题,那将非常有帮助。
任务.py
class SendMessage(Task):
name = "Sending SMS"
max_retries = 10
default_retry_delay = 3
def run(self, message_id, gateway_id=None, **kwargs):
logging.debug("About to send a message.")
try:
message = Message.objects.get(pk=message_id)
except Exception as exc:
raise SendMessage.retry(exc=exc)
if not gateway_id:
if hasattr(message.billee, 'sms_gateway'):
gateway = message.billee.sms_gateway
else:
gateway = Gateway.objects.all()[0]
else:
gateway = Gateway.objects.get(pk=gateway_id)
account = Account.objects.get(user=message.sender)
if account._balance() >= message.length:
response = gateway._send(message)
if response.status == 'Sent':
# Take credit from users account.
transaction = Transaction(
account=account,
amount=- message.charge,
)
transaction.save()
message.billed = True
message.save()
else:
pass
设置.py
# Celery
BROKER_URL = 'amqp://admin:xxxxxx@xx.xxx.xxx.xxx:5672//'
CELERY_SEND_TASK_ERROR_EMAILS = True
阿帕奇配置
<VirtualHost *:80>
ServerName www.domain.com
DocumentRoot /srv/project/domain
WSGIDaemonProcess domain.com processes=2 threads=15 display-name=%{GROUP}
WSGIProcessGroup domain.com
WSGIScriptAlias / /srv/project/domain/apache/django.wsgi
ErrorLog /srv/project/logs/error.log
</VirtualHost>
会议
# Name of nodes to start, here we have a single node
#CELERYD_NODES="w1"
# or we could have three nodes:
CELERYD_NODES="w1 w2 w3"
# Where to chdir at start.
CELERYD_CHDIR="/srv/project/domain"
# How to call "manage.py celeryd_multi"
CELERYD_MULTI="$CELERYD_CHDIR/manage.py celeryd_multi"
# How to call "manage.py celeryctl"
CELERYCTL="$CELERYD_CHDIR/manage.py celeryctl"
# Extra arguments to celeryd
CELERYD_OPTS="--time-limit=900 --concurrency=8"
# %n will be replaced with the nodename.
CELERYD_LOG_FILE="/srv/project/logs/celery/%n.log"
CELERYD_PID_FILE="/srv/project/celery/%n.pid"
# Workers should run as an unprivileged user.
CELERYD_USER="root"
CELERYD_GROUP="root"
# Name of the projects settings module.
export DJANGO_SETTINGS_MODULE="domain.settings"
# Celery Beat Settings.
# Where to chdir at start.
CELERYBEAT_CHDIR="/srv/project/domain"
# Path to celerybeat
CELERYBEAT="$CELERYBEAT_CHDIR/manage.py celerybeat"