Calling pypy functions from C-python

Question

I want to plot CPU intensive functions with matplotlib. I would like to use pypy but it is not compatible with matplotlib. The plotting itself is not CPU intensive and does not need to be accelerated. I wonder if there is a way to call a pypy function from C-python. Could I (ab)use the multiprocessing module and say set_executable("/.../pypy") from C-python?

Answer 1

You could write the output of your computation to stdout and pipe it to another program that reads the data to plot from stdin, like so:

pypy compute.py | python plot.py 

As the intermediary format, you could use a format like JSON which are available in the standard library of both pypy and cpython and can convert from and to python primitives easily.

Alternatively, the compute.py could just use the subprocess module to start plot.py in CPython instead of relying on the pipe being arranged by the shell.

Alternatively, you could use pickle which can preserve more information about python objects, but be careful since pickle isn't a well standardized format.

If you use a recent enough version of CPython and Pypy, another alternative you can try is the multiprocessing.connection module's Listener and Client classes. Note that cross python implementation Listener and Client was broken in some older version of Python and Pypy (multiprocessing Listeners and Clients between python and pypy). The multiprocessing.manager module probably would also work across different cpython and pypy.

Answer 2

If you have CPU intensive functions the best bet is to:

A) See if numpy can help,

B) See if the functions can be optimised in python,

C) Consider writing C extension for your function.

D) Consider embedding cpython and matplotlib in pypy as demonstrated here.

Answer 3

You may use execnet. There are several exmpales of Connecting different Python interpreters