5

我有一个关于进程之间文件句柄的共享资源的问题。这是我的测试代码：

from multiprocessing import Process,Lock,freeze_support,Queue
import tempfile
#from cStringIO import StringIO

class File():
    def __init__(self):
        self.temp = tempfile.TemporaryFile()
        #print self.temp

    def read(self):
        print "reading!!!"
        s = "huanghao is a good boy !!"
        print >> self.temp,s
        self.temp.seek(0,0)

        f_content = self.temp.read()
        print f_content

class MyProcess(Process):
    def __init__(self,queue,*args,**kwargs):
        Process.__init__(self,*args,**kwargs)
        self.queue = queue

    def run(self):
        print "ready to get the file object"
        self.queue.get().read()
        print "file object got"
        file.read()

if __name__ == "__main__":
    freeze_support()
    queue = Queue()
    file = File()

    queue.put(file)
    print "file just put"

    p = MyProcess(queue)
    p.start()

然后我得到KeyError如下所示：

file just put
ready to get the file object
Process MyProcess-1:
Traceback (most recent call last):
  File "D:\Python26\lib\multiprocessing\process.py", line 231, in _bootstrap
    self.run()
  File "E:\tmp\mpt.py", line 35, in run
    self.queue.get().read()
  File "D:\Python26\lib\multiprocessing\queues.py", line 91, in get
    res = self._recv()
  File "D:\Python26\lib\tempfile.py", line 375, in __getattr__
    file = self.__dict__['file']
KeyError: 'file'

我想当我将File()对象放入队列时，对象被序列化，并且文件句柄无法序列化，所以，我得到了KeyError：

有人对此有任何想法吗？如果我想共享具有文件句柄属性的对象，我该怎么办？

4

1 回答 1

7

我必须反对（最终，不会只适合评论；-）@Mark 反复断言文件句柄不能“在正在运行的进程之间传递”——这在真实的现代中根本不是真的操作系统，例如，哦，比如说，Unix（包括免费的 BSD 变体、MacOSX 和 Linux ——嗯，我想知道这个列表中遗漏了哪些操作系统......？-）—— sendmsg当然可以（在“Unix 套接字”上，通过使用SCM_RIGHTS标志）。

现在，不利用这个功能的穷人、有价值multiprocessing的人是完全正确的（即使假设在 Windows 上也可能有黑魔法来实现它）——大多数开发人员无疑会滥用它（让多个进程同时访问同一个打开的文件并运行进入竞争条件）。使用它的唯一正确方法是让具有打开某些文件的独占权限的进程将打开的文件句柄传递给另一个以降低的特权运行的进程——然后再不再使用该句柄本身。无论如何，没有办法在multiprocessing模块中强制执行。

Back to @Andy's original question, unless he's going to work on Linux only (AND with local processes only, too) and willing to play dirty tricks with the /proc filesystem, he's going to have to define his application-level needs more sharply and serialize file objects accordingly. Most files have a path (or can be made to have one: path-less files are pretty rare, actually non-existent on Windows I believe) and thus can be serialized via it -- many others are small enough to serialize by sending their content over -- etc, etc.

于 2009-07-03T02:55:17.400 回答