0

I use a python manager to share data between processes. I wanted to break up my code into smaller pieces and created a function 'soFactory' that would take a dictionary of name/object pairs and register the object with the manager under the given name. In the simple example below I create two lists and register them with the manager. If I use soFactory the manager returns only one of the lists (last one registered) regardless which name I reference. If I unravel the logic of soFactory I get access to the correct object by referencing their registered names. I must be missing something obvious but not seeing it.

# ------------ prototype for SO question -------
from sys import stderr

def soFactory(dictofSo, manager):
    """shared object factory"""
    for n,t in dictofSo.items():
        print >>stderr, 'created item',n,t
        manager.register(n, callable=lambda: t)

def soRoutine(n, t, manager):
    manager.register(n, callable=lambda: t)

def test_soFactory(useFactory=True):
    """tests the soFactory function"""
    from multiprocessing import managers
    m = managers.BaseManager(address='/var/tmp/tqps-test', authkey='abc123')
    mySOlist = {'L1': [1],'L2':[2]}

    if useFactory:
        soFactory(mySOlist, m)
    else:
        for n, t in mySOlist.items():
            soRoutine(n, t, m)

    m.start()

    m.L1().append('only in L1!')
    print >>stderr, m.L1(), m.L2()

>>> test_soFactory()
created item L2 [2]
created item L1 [1]
[1, 'only in L1!'] [1, 'only in L1!']

>>> test_soFactory(useFactory=False)
[1, 'only in L1!'] [2]
>>>
4

1 回答 1

1

这是一个常见的python变量范围陷阱，这是由于lambda创建的闭包记住了变量的名称，而不是对象（或“指针”）。lambda调用时，会在周围的作用域中查找 的值，t此时循环已经结束，所以t分配了它的最终值L1，当然这取决于 的返回值的顺序dictofSo.items()。

您可以通过打开以下调试日志来确认这一点multiprocessing：

import logging
import multiprocessing
multiprocessing.util.log_to_stderr(logging.DEBUG)

L1并L2具有相同的 ID：

[DEBUG/MainProcess] requesting creation of a shared 'L1' object
[DEBUG/BaseManager-1] 'L1' callable returned object with id '104aef878'
[DEBUG/MainProcess] INCREF '104aef878'
...
[DEBUG/MainProcess] requesting creation of a shared 'L2' object
[DEBUG/BaseManager-1] 'L2' callable returned object with id '104aef878'

当你lambda在函数中创建 时soRoutine()，因为一个函数会创建一个新的局部作用域，这一次t可能指向你期望的值。另见这篇文章。

于 2017-11-29T21:05:34.780 回答