0

I'd like people's opinion on which direction to choose between different solutions to implement inter-thread named-pipe communication.

I'm working on a solution for the following: A 3rd party binary on AIX calls a shared object. I build this shared object using the python 2.7.5 api, so I have a python thread (64 bit).

So the stack is: 3rd p binary -> my shared object / dll 'python-bridge' -> python 2.7.5 interpreter (persistent)

From custom code inside the 3rd party binary (in a propriatary language), I initialize the python interpreter through the python-bridge, precompile python code blocks through the python-bridge, and execute these bits of code using PyEval_EvalCode in the bridge. The python interpreter stays alive during the session, and is closed just before the session ends.

Simple sequential python code works fine, and fast. After the call to the shared object method, python references are all decreased (inside the method) and no garbage remains. The precompiled python module stays in memory, works fine. However, I also need to interact with streaming data of the main executable. That executable (of which I don't have the source code) supports fifo through a named pipe, which I want to use for inter-thread communication.

Since the named pipe is blocking, I need a separate thread.

I came up with 3 or 4 alternatives (feel free to give more suggestions)

  1. Use the multiprocess module within python
  2. Make my own C thread, using pthread_create, and use python in there (carefully, I know about the non-threadsafe issues)
  3. Make my own C thread, using pthread_create, parse the named pipe from C, and calling the python interpreter main thread from there
  4. (maybe possible?) use the simpler Threading module of python (which isn't 'pure' threading), and release the GIL at the end of the API call to the bridge. (haven't dared to do this, need someone with insight here. Simple test with Threading and sleep shows it's working within the python call, but the named pipe Thread does nothing after returning to the main non-python process)

What do you suggest?

I'm trying option 1 at the moment, with some success, but it 'feels' a bit bloated to spawn a new process just for parsing a named pipe.

Thanks for your help, Tijs

4

1 回答 1

1

Answering my own question:

I've implemented this (a while back) using option 4. Works good, very stable. Releasing the GIL wasn't happening in my first attempt, because I didn't initialize threading. After that, smooth sailing.

于 2014-03-07T13:21:20.693 回答