Embedding Python in Multi-Threaded c# Applications

Question

My application provide python plugin system.In order to invoke python from c#. I have written a C wrapper of embedding python dll. You can find the gist here C wrapper gist.

Then I use P/Invoke in C# to invoke the C dll. Like this:

[DllImport("PyWinAlfred.dll")]
private extern static IntPtr ExecPython(string directory, string file, string query);

It's ok if I didn't exec above codes in multiple threads created by C#. The problem is if I didn't use thread, main thread in c# will be blocked while the python codes takes a long time to execute.
Here is the C# multiple thread code I use:

ThreadPool.QueueUserWorkItem(state =>
{
   ExecPython(directory,file,query);
});

So, I tried to add multiple thread support in C wrapper code. As you can see in that gist. I tried to add:

// Create GIL/enable threads
PyEval_InitThreads();
PyGILState_STATE gstate = PyGILState_Ensure();

//python codes

PyGILState_Release(gstate);

But this doesn't work for me. When I create some threads to invoke that dll, I will get an exception.

I also tried following codes. Still didn't work.

PyThreadState* state = PyThreadState_Get();
// Once in each thread
PyThreadState* stateForNewThread = PyThreadState_New(state->interp);
PyEval_RestoreThread(stateForNewThread);

//python codes

PyEval_SaveThread();

I was wondering if this is because I put Py_Initialize(); in every invoke. When thread2 execute the Py_Initialize(); will clear all infos that thread1 may requires. I tried to invoke Py_Initialize() when my application starts, so it will be invoke only once. But didn't work. Looks like these two calls are totally isolated in p/invoke calls.

I'm poor in C and embedding python, does anyone has related experience? My project hosts in https://github.com/qianlifeng/WinAlfred If you guys want to see the codes I described above.

Answer 1

Finally, I find http://pythonnet.github.io/. You can find details in this project WinAlfred. I solved the multiplethread issues in this project.