c++ - 解释 Python 扩展多线程

Question

Python解释器有一个全局解释器锁，我的理解是扩展必须在多线程环境中获取它。但是Boost.Python HOWTO 页面说扩展功能必须释放 GIL 并在退出时重新获取它。

我想抵制在这里猜测的诱惑，所以我想知道以下场景中的 GIL 锁定模式应该是什么：

1. 从 python 调用扩展（可能在 python 线程中运行）。
2. 并且扩展的后台线程回调Py_*函数。

最后一个问题是，为什么链接的文件说 GIL 应该被释放并重新获得？

Answer 1

Whenever Python is interpreting bytecode the GIL is being held by the currently running thread. No other Python thread can run until it manages to acquire the GIL.

When the interpreter has called into native code that code has two options regarding the GIL:

1. It could do nothing at all.
2. It could release the GIL while it works and then reacquire it before returning to Python

If the native code makes a lot of calls back to Python's runtime it should take option 1: you can never call Python's runtime safely unless you hold the GIL (with a few exceptions such as making the call to acquire the GIL if you don't have it).

If the native code does a lot of work which doesn't involve Python in any way then you can use option 2: as soon as you release the GIL Python will be able to schedule another Python thread so you get some parallelism. If you don't release the GIL then none of Python's other threads can execute while your Boost code is running: that's why the docs tell you to release and reacquire the GIL.

If you go this way then you must be careful to make all your access to Py_* functions before you release the GIL or after you reacquire it. This may mean you have to make local copies of data as you cannot safely access Python data types such as list or dictionary elements while the GIL is released.

If your Boost code needs to call back into Python while the GIL is released then you need to acquire the GIL, make the call, release the GIL. Try to avoid making such calls from threads that weren't created by Python as they need additional work to be able to acquire the GIL.