Compiling required external modules with cython

Question

I'm building a standalone executable using Cython on Linux.

I have the following code:

import psycopg2 as pg

conn = pg.connect('dbname=**** user=**** password=****')
cur = conn.cursor()
cur.execute('SELECT version()')
print(cur.fetchone())

The problem is when the machine does not have the Python package psycopg2 installed, throws the following exception:

Traceback (most recent call last):
File "test.py", line 2, in init test (test.c:872)
    import psycopg2 as pg
ImportError: No module named 'psycopg2'

Im building using the --embed cython flag.

How can I make Cython to compile that particular package too?

Answer 1

From my experience, it is not that straightforward to create a standalone executable from multiples python files (yours or from dependencies like psycopg2). I would say there are a couple of approaches here I would try:

The first one would be cython_freeze https://github.com/cython/cython/tree/master/Demos/freeze I do not use it myself, so I cannot tell much.

The second one is to use pyinstaller to create such executable. It takes as input the .py or .pyc files and embed them into one executable, together with the python interpreter and required dependencies, so you don't have to install anything on the target machine. Note, however, that your code will run as interpreted python and can be easily decompiled and inspected.

If you really need to compile (cythonize) your code, then you can first cythonize() and the build with setup() your extensions, then run pyinstaller as above (taking care that it doesnt find the .py or .pyc files, just the .pyd or .so extensions) to generate the standalone executable. In both cases, pyinstaller will collect all your dependencies and embed them in the executable (even if it fails, you can tell pyinstaller to embed them with hidden_imports).

There are surely other approaches, like py2exe, but when I researched and played with several technologies some months ago, pyinstaller was the best option for me. I do the process in win, linux and mac without many changes.

EDIT: I didn't realize that the example is python 3. Pyinstaller only works for 2.x now.

Answer 2

--embed means the Python interpreter is embedded in your executable. It does not mean independence from Python. It does not do what you think. It sounds more like you need a tool like py2exe, py2app or pyfreeze.

Answer 3

Nuitka is the tool that you need.

You feed it your Python app, it does a lot of clever things, and spits out an executable or extension module.

Right now Nuitka is a good replacement for the Python interpreter and compiles every construct that CPython 2.6, 2.7, 3.2, 3.3 and 3.4 offer. It translates the Python into a C++ program that then uses "libpython" to execute in the same way as CPython does, in a very compatible way.

It is somewhat faster than CPython already, but currently it doesn't make all the optimizations possible, but a 258% factor on pystone is a good start (number is from version 0.3.11).