3

在我的映射器中，上下文被返回为 NULL

class DeleteOldObservationsMapper(object):
  """Mapper for deleting old observations."""

  def __init__(self):
      logging.info('DeleteOldObservationsMapper init')
      ctx = mapreduce.context.get()
      when = ctx.mapreduce_spec.mapper.params.get('before_timestamp_seconds')
      assert when
      self.before_datetime = datetime.datetime.utcfromtimestamp(when)
      logging.info('before_datetime %s', self.before_datetime)

这是错误跟踪：

ERROR    2013-05-24 16:03:38,662 webapp2.py:1552] 'NoneType' object has no attribute 
'mapreduce_spec'
Traceback (most recent call last):
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/Users/leoromanovsky/code/adapt/server/mapreduce/base_handler.py", line 66, in post
    self.handle()
  File "/Users/leoromanovsky/code/adapt/server/mapreduce/handlers.py", line 320, in handle
    tstate = model.TransientShardState.from_request(self.request)
  File "/Users/leoromanovsky/code/adapt/server/mapreduce/model.py", line 993, in from_request
    handler = mapreduce_spec.mapper.handler
  File "/Users/leoromanovsky/code/adapt/server/mapreduce/model.py", line 618, in get_handler
    return util.handler_for_name(self.handler_spec)
  File "/Users/leoromanovsky/code/adapt/server/mapreduce/util.py", line 149, in handler_for_name
    return getattr(resolved_name.im_class(), resolved_name.__name__)
  File "/Users/leoromanovsky/code/adapt/server/jobs.py", line 22, in __init__
    when = ctx.mapreduce_spec.mapper.params.get('before_timestamp_seconds')
AttributeError: 'NoneType' object has no attribute 'mapreduce_spec'
4

2 回答 2

2

我们最近在一个项目中遇到了这个问题threadsafe: false，并且能够通过更改导入context模块的方式来解决这个问题。Kevin 在这个错误报告中很好地解释了这个问题。

kevin.mo...@gmail.com 也许这对其他人有帮助，但我看到了这个问题，只是为我的用例解决了这个问题。在我们的项目中，我们在子文件夹“libs/external/mapreduce”中有 mapreduce 库，而不是我们项目的根目录。

库从自身导入上下文（除其他外）为from mapreduce import context. 因此，为了使其工作，我们使用如下导入操作：

import os, sys 
sys.path.append(os.path.join(os.path.dirname(__file__), 'libs/external'))

然而，在我们的代码中的一些地方，我们仍然像这样导入上下文：

from libs.external.mapreduce import context

这实际上会导致 context 被导入两次并被 mapreduce.context.Context._local设置两次，因此有两个不同的thread._local. 当Context._set被调用来存储上下文以供以后在映射器函数或其他东西中检索时，它将使用原始thread._local实例。

稍后当我们的映射器模块再次导入上下文时，它会获得一个新 thread._local实例。当我们那时calling context.get()它正在返回实际上没有上下文的新实例：

AttributeError: 'NoneType' object has no attribute 'mapreduce_spec'

将我们所有的导入转换from mapreduce import context 为为我们修复它。

于 2013-11-28T18:53:15.673 回答
1

我认为这是您问题的答案：

http://code.google.com/p/appengine-mapreduce/issues/detail?id=127

mapreduce.context.get() 根本不是线程保存...

所以你可以做的是将它打包到一个包装器中，然后通过使用锁定机制使其成为线程保存。

于 2013-07-01T20:40:07.650 回答