2

我有一个 Django 项目,将huey 作为任务队列。在我的开发环境中(见下文),一切都按预期工作:

  1. [保存] Django 模型实例保存
  2. [触发] huey 任务触发
  3. [execute] huey 任务通过运行 consumer ( run_huey)来执行

但是我无法在我的生产环境中得到第三点 [执行]。我能发现的唯一区别是,消费者run_huey是由一个 systemd 服务单元启动的,而网络服务器是一个 Apache2(见下文)。DEBUG = True如果我在我的生产环境中设置也没关系。

消费者确实正确识别了db_taskschedule_virusscan”,但没有接收/接收由我的模型保存方法触发的任何任务(此处schedule_virusscan(self.id):)

immediate=False用于我的 Huey 实例,因为我run_huey什至在我的开发环境中运行。

我得到这种行为huey.FileHueyhuey.SqliteHuey

问题

我的消费者(仅在生产中)没有执行甚至接收任何任务,我错过了什么?

设置

网络服务器

  • Debian Buster 上的 apache (2.4) 和 mod_wsgi (4.6.5)
  • run_huey并且 WSGIDaemonProcess 确实以相同(专用,非 root)用户身份运行

Django 项目

# settings.py

DEBUG = False  # True in dev environment

HUEY = {
    'huey_class': 'huey.FileHuey',
    'path': '/tmp/huey',
    'immediate': False,
    'immediate_use_memory': False,
    'consumer': {
        'workers': 2,  # use 2 threads
    },
}
# my-huey-systemd-service-unit.service

[Unit]
After=network.target

[Service]
Restart=always
WorkingDirectory=/path/to/project
ExecStart=/path/to/venv/bin/python3 \
    /path/to/project/manage.py run_huey \
    --logfile=/path/to/project/wagtailapiforms-huey.log
ExecReload=/bin/kill -s SIGHUP $MAINPID
ExecStop=/bin/kill -s SIGINT $MAINPID

[Install]
WantedBy=default.target
# project/apps/core/tasks.py

from huey.contrib.djhuey import db_task

@db_task()
def schedule_virusscan(attachment_id):
    print(f"Scan for virus for attachment with pk {attachment_id}...")
    ...
# project/apps/core/models.py

class Attachment(models.Model):
    ...
    def save(self, *args, **kwargs):
        from .tasks import schedule_virusscan
        super().save()
        print(f"Attachment instance {self.id}: trigger schedule_virusscan...")
        _scan = schedule_virusscan(self.id)
        # _scan()
# prints from server

[Tue Jun 29 2021] [wsgi:error] [pid 4825] [remote 10.1.x.x:36754] Attachment instance created, calling super().save()...
[Tue Jun 29 2021] [wsgi:error] [pid 4825] [remote 10.1.x.x:36754] Attachment instance 6b0dd19d-377d-43b6-9d9a-343c80793447: trigger schedule_virusscan...
[Tue Jun 29 2021] [wsgi:error] [pid 4825] [remote 10.1.x.x:36754] _scan:   <Result: task 22306b18-d45e-4458-8052-138be9c996f1>
# prints from run_huey
# /path/to/project/wagtailapiforms-huey.log

[2021-06-29] INFO:huey.consumer:MainThread:Huey consumer started with 2 thread, PID 4494 at 2021-06-29 18:48:06.834951
[2021-06-29] INFO:huey.consumer:MainThread:Scheduler runs every 1 second(s).
[2021-06-29] INFO:huey.consumer:MainThread:Periodic tasks are enabled.
[2021-06-29] INFO:huey.consumer:MainThread:The following commands are available:
+ project.apps.core.tasks.schedule_virusscan

# Here I expected the output of the execution of received tasks - but on production this consumer does not receive anything.
# In my development environment this looks like:
# [2021-06-29 INFO:huey:Worker-2:Executing project.apps.core.tasks.schedule_virusscan: 4f877b2e-559a-43c9-9bf3-5821ea426842
# Huey task schedule_virusscan: Scan for virus for attachment with pk 38950ef9-469a-485c-afee-24cbc6fcef12...
# [2021-06-29] INFO:huey:Worker-2:project.apps.core.tasks.schedule_virusscan: 4f877b2e-559a-43c9-9bf3-5821ea426842 executed in 0.109s
4

1 回答 1

2

该问题与 huey 或 mod_wsgi 无关,而是与 hueySqliteHueyFileHuey路径设置的位置以及 apache2 systemd 服务单元PrivateTmp=true(参见/lib/systemd/system/apache2.service,假设 Debian 10)有关。

解决方案:/tmp不在 global 下方,而是在单独的目录中 找到 huey 数据库/文件 ,消费者 ( run_huey) 以及 webapp 都可以访问该目录。

PrivateTmp=true(只是一个选择)的含义:

于 2021-08-12T11:18:07.363 回答