2

我有一个空闲的后台进程来处理队列中的数据，我通过以下方式实现了它。此示例中传递的数据只是一个整数，但我将传递最多 1000 个整数的列表，并每秒将最多 100 个列表放入队列中。这是正确的方法，还是我应该查看更复杂的 RPC 和服务器方法？

import multiprocessing
import Queue
import time

class MyProcess(multiprocessing.Process):

    def __init__(self, queue, cmds):
        multiprocessing.Process.__init__(self)  
        self.q = queue
        self.cmds = cmds

    def run(self):
        exit_flag = False
        while True:
            try:
                obj = self.q.get(False)
                print obj
            except Queue.Empty:
                if exit_flag:
                    break
                else:
                    pass
            if not exit_flag and self.cmds.poll():
                cmd = self.cmds.recv()
                if cmd == -1:
                    exit_flag = True
            time.sleep(.01)

if __name__ == '__main__':
    queue = multiprocessing.Queue()
    proc2main, main2proc = multiprocessing.Pipe(duplex=False)
    p = MyProcess(queue, proc2main)
    p.start()
    for i in range(5):
        queue.put(i)    
    main2proc.send(-1)
    proc2main.close()
    main2proc.close()
    # Wait for the worker to finish
    queue.close()
    queue.join_thread()
    p.join()
4

1 回答 1

0

It depends on how long it will take to process the data. I can't tell because I don't have a sample of the data, but in general it is better to move to more elaborate RPC and server methods when you need things like load balancing, guaranteed uptime, or scalability. Just remember that these things will add complexity, which may make your application harder to deploy, debug, and maintain. It will also increase the latency that it takes to process a task (which might or might not be a concern to you).

I would test it with some sample data, and determine if you need the scalability that multiple servers provide.

于 2012-07-09T09:29:39.497 回答