How to use .so modules from Python 2 in Python 3?

Question

probably my question is obvious but I was not able to find an obvious decision.

There are Python 2.6+ extensions called audit and auparse. These are dynamical libraries distributed with audit-libs-python package:

[vitaly@thermaltake tmp]$ repoquery -lq audit-libs-python
/usr/lib64/python2.7/site-packages/_audit.so
/usr/lib64/python2.7/site-packages/audit.py
/usr/lib64/python2.7/site-packages/audit.pyc
/usr/lib64/python2.7/site-packages/audit.pyo
/usr/lib64/python2.7/site-packages/auparse.so

I would like to use this extension in the up-to-date Python interpreter because of suspicions about the incorrect handling of dynamic memory in python 2.6+. For some reason I cannot load them from Python 3.3:

[vitaly@thermaltake ~]$ python3.3
Python 3.3.2 (default, Mar  5 2014, 08:21:05) 
[GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append("/usr/lib64/python2.7/site-packages/")
>>> import auparse
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /usr/lib64/python2.7/site-packages/auparse.so: undefined symbol: _Py_ZeroStruct
>>> import audit
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.7/site-packages/audit.py", line 28, in <module>
    _audit = swig_import_helper()
  File "/usr/lib64/python2.7/site-packages/audit.py", line 24, in swig_import_helper
    _mod = imp.load_module('_audit', fp, pathname, description)
  File "/usr/lib64/python3.3/imp.py", line 183, in load_module
    return load_dynamic(name, filename, file)
ImportError: /usr/lib64/python2.7/site-packages/_audit.so: undefined symbol: PyInstance_Type

I would be glad if anyone could clarify the procedure of importing the modules of this kind into the modern Python interpreter. It's hard to believe that backward compatibility between 2nd and 3rd branches was broken in this case too. Thank you.

Answer 1

.so modules have to be compiled for each specific Python version - you can't even reuse an .so module built for Python 2.6 with Python 2.7.

When crossing over to Python 3 it gets worse, since there are some API changes, and the SO simply won't build unchanged from the .C file (with possible exceptions).

One workaround is to serve the functions you want to use in the 2.6 module with xmlrpc, then call then from a separate Python process runing Python 3.x - that should be the simplest way.