我有multiProcessing.Process
目标函数采用输入和输出队列的对象。
他们将一些数据放入输出队列,这是一个带有内部指针的包装 ctypes 结构。当然,pickle
应该序列化数据的模块会中断:
ValueError:不能腌制包含指针的 ctypes 对象
我可以以某种方式从我的子进程中获取带有指针的 ctypes 结构而不将它们转储到文件中吗?
代码如下
# -*- coding: utf-8 -*-
import multiprocessing as mp
from liblinear import *
from liblinearutil import *
def processTarget(inQueue, outQueue):
while(not inQueue.empty()):
inVal = inQueue.get()
#training model
y, x = [1,-1], [{1:inVal, 3:3*inVal}, {1:-1,3:-1}]
prob = problem(y, x)
param = parameter('-c 4 -B 1')
m = train(prob, param)
outQueue.put((inVal * 2, m))
print "done", inVal
inQueue.task_done()
def Main():
processes = []
inQueue = mp.JoinableQueue()
for i in xrange(10):
inQueue.put(i)
outQueue = mp.JoinableQueue()
for i in xrange(5):
process = mp.Process(target=processTarget, args=(inQueue, outQueue))
print "starting", i
process.start()
print "started", i
inQueue.join()
print "JOINED"
while(not outQueue.empty()):
print outQueue.get()
if __name__ == '__main__':
Main()