1

跟进这个问题pathos multiprocessing cannot pickle

运行代码在一个：

from pathos.multiprocessing import ProcessingPool as Pool

def f(x):
    return x*x

if __name__ == '__main__':
    p = Pool(5)
    print(p.map(f, [1,2,3]))

Traceback (most recent call last):
  File "\\femto.niddk.nih.gov\C\All Projects\NMR High Pressure\Software\multiprocessing\multiprocessing_playground_2.py", line 12, in <module>
    results = ProcessingPool().map(b.boo, [[12,3,456],[8,9,10],['a','b','cde']])
  File "C:\Python27\lib\site-packages\pathos\multiprocessing.py", line 137, in map
    return _pool.map(star(f), zip(*args)) # chunksize
  File "C:\Python27\lib\site-packages\multiprocess\pool.py", line 251, in map
    return self.map_async(func, iterable, chunksize).get()
  File "C:\Python27\lib\site-packages\multiprocess\pool.py", line 567, in get
    raise self._value
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed
>>>

这是更多信息：

>>> pathos.__version__
'0.2.2.dev0'
>>> import multiprocess
>>> multiprocess.__version__
'0.70.5'
>>> import multiprocessing
>>> multiprocessing.__version__
'0.70a1'
>>> multiprocess.Pool().map(lambda x:x*x, range(10))

Traceback (most recent call last):
  File "<pyshell#11>", line 1, in <module>
    multiprocess.Pool().map(lambda x:x*x, range(10))
  File "C:\Python27\lib\site-packages\multiprocess\pool.py", line 251, in map
    return self.map_async(func, iterable, chunksize).get()
  File "C:\Python27\lib\site-packages\multiprocess\pool.py", line 567, in get
    raise self._value
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

到底是怎么回事？为什么不工作？Pathos 库不应该使用莳萝而不是酸洗吗？

4

1 回答 1

2

如果我使用它会起作用

from pathos.pools import ParallelPool as Pool

代替

from pathos.multiprocessing import ProcessingPool as Pool

的

from pathos.pools import ProcessPool as Pool

谁能解释一下有什么区别？所有这些方法都会创建不同的过程。

于 2018-06-07T18:32:08.953 回答