使用multiprocessing.imap_unordered。它允许您启动一个进程池,将作业发送到池,并在它们进入时接收结果。
这是下载一堆 URL 的示例代码:
import multiprocessing, re, subprocess, sys
CMD_LIST = [
["wget", "-qO-", "http://ipecho.net/plain"],
["curl", '-s', "http://www.networksecuritytoolkit.org/nst/cgi-bin/ip.cgi"],
["curl", '-s', "v4.ident.me"],
["curl", '-s', "ipv4.icanhazip.com"],
["curl", '-s', "ipv4.ipogre.com"],
]
ip_pat = re.compile('[0-9.]{7,}')
pool = multiprocessing.Pool(5)
for output in pool.imap_unordered(subprocess.check_output, CMD_LIST):
print 'output:',output
m = ip_pat.search(output)
if m:
print 'GOT IP:', m.group(0)
pool.terminate()
sys.exit(0)
print 'no IP found'