我正在为 Django、Celery、djcelery 和 PeriodicTasks 苦苦挣扎。
我创建了一个任务来为 Adsense 提取报告以生成实时统计报告。这是我的任务:
import datetime
import httplib2
import logging
from apiclient.discovery import build
from celery.task import PeriodicTask
from django.contrib.auth.models import User
from oauth2client.django_orm import Storage
from .models import Credential, Revenue
logger = logging.getLogger(__name__)
class GetReportTask(PeriodicTask):
run_every = datetime.timedelta(minutes=2)
def run(self, *args, **kwargs):
scraper = Scraper()
scraper.get_report()
class Scraper(object):
TODAY = datetime.date.today()
YESTERDAY = TODAY - datetime.timedelta(days=1)
def get_report(self, start_date=YESTERDAY, end_date=TODAY):
logger.info('Scraping Adsense report from {0} to {1}.'.format(
start_date, end_date))
user = User.objects.get(pk=1)
storage = Storage(Credential, 'id', user, 'credential')
credential = storage.get()
if not credential is None and credential.invalid is False:
http = httplib2.Http()
http = credential.authorize(http)
service = build('adsense', 'v1.2', http=http)
reports = service.reports()
report = reports.generate(
startDate=start_date.strftime('%Y-%m-%d'),
endDate=end_date.strftime('%Y-%m-%d'),
dimension='DATE',
metric='EARNINGS',
)
data = report.execute()
for row in data['rows']:
date = row[0]
revenue = row[1]
try:
record = Revenue.objects.get(date=date)
except Revenue.DoesNotExist:
record = Revenue()
record.date = date
record.revenue = revenue
record.save()
else:
logger.error('Invalid Adsense Credentials')
我正在使用 Celery 和 RabbitMQ。这是我的设置:
# Celery/RabbitMQ
BROKER_HOST = "localhost"
BROKER_PORT = 5672
BROKER_USER = "myuser"
BROKER_PASSWORD = "****"
BROKER_VHOST = "myvhost"
CELERYD_CONCURRENCY = 1
CELERYD_NODES = "w1"
CELERY_RESULT_BACKEND = "amqp"
CELERY_TIMEZONE = 'America/Denver'
CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'
import djcelery
djcelery.setup_loader()
乍一看,一切似乎都正常,但在打开记录器并观察它运行后,我发现它至少连续四次运行该任务 - 有时更多。它似乎也每分钟运行一次,而不是每两分钟运行一次。我尝试将 run_every 更改为使用 crontab,但我得到了相同的结果。
我正在使用主管开始 celerybeat。这是我使用的命令:
python manage.py celeryd -B -E -c 1
关于为什么它没有按预期工作的任何想法?
哦,还有一件事,在日期变化之后,它会继续使用它第一次运行的日期范围。因此,随着时间的推移,它会继续获取任务开始运行当天的统计信息 - 除非我在某个时候手动运行任务,否则它会更改为我上次手动运行它的日期。有人能告诉我为什么会这样吗?