2

我正在运行 Django 1.4.5、Celery 3.0.15、Django Celery 3.0.11、Johnny Cache 1.4。

celery 任务中对 ORM 的调用有时会失败，并出现奇怪的错误，例如invalid literal for int() with base 10: 'a'"or <MaybeEncodingError: Error sending result: ''<ExceptionInfo: ObjectDoesNotExist()>''. Reason: ''PicklingError("Can\'t pickle <class \'scsite.models.DoesNotExist\'>: attribute lookup scsite.models.DoesNotExist failed",)''.>：

这是一个示例堆栈跟踪：

Task scsite.tasks.send_signup_email[aecf0561-65af-4d11-a3a0-63d88b7e1a70] raised exception: ValueError("invalid literal for int() with base 10: 'd'",)

Task scsite.tasks.send_signup_email[aecf0561-65af-4d11-a3a0-63d88b7e1a70] raised exception: ValueError("invalid literal for int() with base 10: 'd'",)

Stacktrace (most recent call last):

  File "celery/execute/trace.py", line 192, in trace_task
    R = I.handle_error_state(task, eager=eager)
  File "scsite/tasks.py", line 537, in send_signup_email
    email_html = email_template.render(Context(data))
  File "django/template/base.py", line 142, in render
    context.render_context.pop()
  File "django/template/base.py", line 134, in _render
    return self.nodelist.render(context)
  File "django/template/base.py", line 823, in render
    bit = self.render_node(node, context)
  File "django/template/base.py", line 837, in render_node
    return node.render(context)
  File "django/template/loader_tags.py", line 155, in render
    return self.render_template(self.template, context)
  File "django/template/loader_tags.py", line 137, in render_template
    output = template.render(context)
  File "django/template/base.py", line 142, in render
    context.render_context.pop()
  File "django/template/base.py", line 134, in _render
    return self.nodelist.render(context)
  File "django/template/base.py", line 823, in render
    bit = self.render_node(node, context)
  File "django/template/base.py", line 837, in render_node
    return node.render(context)
  File "django/template/defaulttags.py", line 192, in render
    nodelist.append(node.render(context))
  File "django/template/defaulttags.py", line 474, in render
    self.extra_context.iteritems()])
  File "django/template/base.py", line 584, in resolve
    obj = settings.TEMPLATE_STRING_IF_INVALID
  File "django/template/base.py", line 721, in resolve
    value = self._resolve_lookup(context)
  File "django/template/base.py", line 781, in _resolve_lookup
    raise
  File "django/db/models/manager.py", line 119, in count
    return self.get_query_set().count()
  File "django/db/models/fields/related.py", line 461, in get_query_set
    return super(RelatedManager, self).get_query_set().using(db).filter(**self.core_filters)
  File "django/db/models/query.py", line 624, in filter
    return self._filter_or_exclude(False, *args, **kwargs)
  File "django/db/models/query.py", line 642, in _filter_or_exclude
    clone.query.add_q(Q(*args, **kwargs))
  File "django/db/models/sql/query.py", line 1250, in add_q
    can_reuse=used_aliases, force_having=force_having)
  File "django/db/models/sql/query.py", line 1185, in add_filter
    connector)
  File "django/db/models/sql/where.py", line 69, in add
    value = obj.prepare(lookup_type, value)
  File "django/db/models/sql/where.py", line 320, in prepare
    return self.field.get_prep_lookup(lookup_type, value)
  File "django/db/models/fields/__init__.py", line 310, in get_prep_lookup
    return self.get_prep_value(value)
  File "django/db/models/fields/__init__.py", line 537, in get_prep_value
    return int(value)




Task scsite.tasks.update_various_denorm_fields[edddb260-041d-4195-83d7-f2e731e5d1a5] raised exception: ValueError("invalid literal for int() with base 10: 'a'",)

Stacktrace (most recent call last):

  File "celery/task/trace.py", line 242, in trace_task
    R = I.handle_error_state(task, eager=eager)
  File "celery/task/trace.py", line 415, in __protected_call__
    return self.run(*args, **kwargs)
  File "scsite/tasks.py", line 470, in update_various_denorm_fields
    Person.objects.update_question_counts()
  File "scsite/managers.py", line 715, in update_question_counts
    person.update_question_count_denorm()
  File "scsite/models.py", line 2712, in update_question_count_denorm
    self.question_count_denorm = self.questions.count()
  File "django/db/models/manager.py", line 119, in count
    return self.get_query_set().count()
  File "django/db/models/fields/related.py", line 567, in get_query_set
    return super(ManyRelatedManager, self).get_query_set().using(db)._next_is_sticky().filter(**self.core_filters)
  File "django/db/models/query.py", line 624, in filter
    return self._filter_or_exclude(False, *args, **kwargs)
  File "django/db/models/query.py", line 642, in _filter_or_exclude
    clone.query.add_q(Q(*args, **kwargs))
  File "django/db/models/sql/query.py", line 1250, in add_q
    can_reuse=used_aliases, force_having=force_having)
  File "django/db/models/sql/query.py", line 1185, in add_filter
    connector)
  File "django/db/models/sql/where.py", line 69, in add
    value = obj.prepare(lookup_type, value)
  File "django/db/models/sql/where.py", line 320, in prepare
    return self.field.get_prep_lookup(lookup_type, value)
  File "django/db/models/fields/__init__.py", line 310, in get_prep_lookup
    return self.get_prep_value(value)
  File "django/db/models/fields/__init__.py", line 537, in get_prep_value
    return int(value)

我相信错误不在我的代码中，看起来缓存由于某种原因损坏了，并且发生了以下情况：

  • 检索 pk=123 的 CustomModel。
  • 它被传递给 Celery 进行处理。
  • 出于某种原因，Custom 模型的 pk 变成了 'a'、'd' 之类的字符，而不是 123。另一个调用是 withMyModel.objects.get(pk='a')并且显然失败了。

实际上，重新访问此代码，似乎我并没有像上面提到的那样将对象传递给 Celery，只是从一个简单的 celery 任务中运行以下管理器方法。

@task
def mytask():
   Person.objects.update_question_count()


class PersonManager(models.Manager):
   def update_question_counts(self): 
        persons = self.all() # 1
        for person in persons: 
           person.update_question_count_denorm() #3

我的猜测如下：当第 1 行运行时，人员是从缓存中获取的，但在处理第 3 行时，第 1 行中获得的缓存已损坏。

4

0 回答 0