2

当前代码是：

def export_data(file):
    <runs the db2 database command to export tables to file>

def export_to_files(yaml):
    logger = logging.getLogger("export_to_files")
    thread1 = threading.Thread(target=export_data, args=[out_file1])
    thread1.start()
    thread2 = threading.Thread(target=export_data, args=[out_file2])
    thread2.start()
    thread1.join()
    thread2.join()

def main():
    export_to_files()

if __name__ == "__main__":
    main()

我的理解是join()只会阻塞调用线程。但是，我没有意识到这thread1.join()甚至会阻止thread2执行，基本上使代码只运行 1 个线程，即thread1.

如何同时执行两个线程，同时让主线程等待两个线程完成？

编辑：我的立场是正确的，2个线程确实运行，但似乎只有1个线程实际上在某个时间点“做”事情。

更详细地说，callable_method是从数据库中读取数据并写入文件。虽然我现在可以看到 2 个文件正在更新（每个线程写入一个单独的文件），但其中一个文件已经有一段时间没有更新了，而另一个文件是当前时间的最新文件。

没有使用连接对象。查询从 db2 命令行界面运行。

4

3 回答 3

3

您可以使用大部分未记录的ThreadPool类multiprocessing.pool来执行以下操作：

from multiprocessing.pool import ThreadPool
import random
import threading
import time

MAX_THREADS = 2
print_lock = threading.Lock()

def export_data(fileName):
    # simulate writing to file
    runtime = random.randint(1, 10)
    while runtime:
        with print_lock: # prevent overlapped printing
            print('[{:2d}] Writing to {}...'.format(runtime, fileName))
        time.sleep(1)
        runtime -= 1

def export_to_files(filenames):
    pool = ThreadPool(processes=MAX_THREADS)
    pool.map_async(export_data, filenames)
    pool.close()
    pool.join()  # block until all threads exit

def main():
    export_to_files(['out_file1', 'out_file2', 'out_file3'])

if __name__ == "__main__":
    main()

示例输出：

[ 9] Writing to out_file1...
[ 6] Writing to out_file2...
[ 5] Writing to out_file2...
[ 8] Writing to out_file1...
[ 4] Writing to out_file2...
[ 7] Writing to out_file1...
[ 3] Writing to out_file2...
[ 6] Writing to out_file1...
[ 2] Writing to out_file2...
[ 5] Writing to out_file1...
[ 1] Writing to out_file2...
[ 4] Writing to out_file1...
[ 8] Writing to out_file3...
[ 3] Writing to out_file1...
[ 7] Writing to out_file3...
[ 2] Writing to out_file1...
[ 6] Writing to out_file3...
[ 1] Writing to out_file1...
[ 5] Writing to out_file3...
[ 4] Writing to out_file3...
[ 3] Writing to out_file3...
[ 2] Writing to out_file3...
[ 1] Writing to out_file3...
于 2013-08-16T23:12:30.770 回答
0

您的可见代码很好，但是我们不可见的一些代码确实使用了锁定，锁定甚至可以在数据库本身中发生。

于 2013-08-17T15:28:35.057 回答
0

这说明了示例代码的可运行版本：

import time
import threading

def export_data(fileName):
    # runs the db2 database command to export tables to file
    while True:
        print 'If I were the real function, I would be writing to ' + fileName
        time.sleep(1)

thread1 = threading.Thread(target=export_data, args=[ 'out_file1' ])
thread2 = threading.Thread(target=export_data, args=[ 'out_file2' ])

thread1.start()
thread2.start()

thread1.join()
thread2.join()
于 2013-08-16T20:50:24.613 回答