2

我有一个正在监视 Web 服务请求工作区的 python 脚本。每次客户向我的 Web 服务提交作业时,都会在众所周知的位置创建一个唯一的作业文件夹。我有一个脚本在没有“标志”的情况下轮询这个众所周知的文件夹位置(一个具有特定名称的空白文本文件,表示该作业的处理已完成)。

现在我的脚本可以调用工作脚本来处理新文件夹的内容,但必须等到工作脚本完成才能继续分发文件夹。

我的问题是有哪些选项可以使工作脚本的新实例实例化并将控制权返回给经理。创建一个接受工作脚本参数并让管理器脚本通过命令行调用它的python可执行文件是否可行?或者将工作脚本创建到一个可以有许多 instation 处理工作的类中?

一旦工作脚本完成,它就不需要向管理器脚本作业完成发送消息。它将通过将文本文件放入目录中来完成此操作。虽然现在我想起来了,但我必须在某个地方保存每个作业目录已经分发,因为处理工作脚本需要 1.5 分钟。

任何建议/链接将不胜感激。

4

1 回答 1

0

First of all, I agree that you need to put a flag in your directories indicating that a directory is being processed. The master script should be the only one to set the flag, or you will risk race conditions (two worker scripts taking the same directory at the same time). You can use the same file; the master script creates it empty (meaning "in progress") and the worker script writes 1B in it (meaning "done"). That way, the master script only has to check the existence of the flag.

Back to your question:

  • you can indeed make your worker script into a standalone program, and call it via the subprocess module;

  • you can make it a thread (with the threading module [2]), which is somewhat easier to code; this may be inefficient because of the GIL, but if your worker script is highly IO-bound, it should not be too much of a problem;

  • if you are using Python 3, you may want to look at the multiprocessing module [3]
    which I never used but seems to mix the usability of threading without being vulnerable to the GIL; it seems that it is not completely portable though.

Hope this helps

  • [1] http://docs.python.org/library/subprocess.html
  • [2] http://docs.python.org/library/threading.html
  • [3] http://docs.python.org/dev/library/multiprocessing.html
于 2012-04-10T15:18:00.707 回答