How to build Python C extension modules with autotools

Question

Most of the documentation available for building Python extension modules uses distutils, but I would like to achieve this by using the appropriate python autoconf & automake macros instead.

I'd like to know if there is an open source project out there that does exactly this. Most of the ones I've found end up relying on a setup.py file. Using that approach works, but unfortunately ends up rebuilding the entire source tree any time I make a modification to the module source files.

Answer 1

All PyGTK extensions use autotools, so if the PyGTK aspects don't kill the whole thing for you, it might be worth having a look at the PyGTK source code. Additionally, here is one I wrote which is more simple.

Answer 2

Supposing that you have a project with a directory called src, so let's follow the follow steps to get a python extension built and packaged using autotools:

Create the Makefile.am files

First, you need to create one Makefile.am in the root of your project, basically (but not exclusively) listing the subdirectories that should also be processed. You will end up with something like this:

SUBDIRS = src

The second one, inside the src directory will hold the instructions to actually compile your python extension. It will look like this:

myextdir = $(pkgpythondir)
myext_PYTHON = file1.py file2.py

pyexec_LTLIBRARIES = _myext.la

_myext_la_SOURCES = myext.cpp
_myext_la_CPPFLAGS = $(PYTHON_CFLAGS)
_myext_la_LDFLAGS = -module -avoid-version -export-symbols-regex initmyext
_myext_la_LIBADD = $(top_builddir)/lib/libhollow.la

EXTRA_DIST = myext.h

Write the configure.ac

This file must be created in the root directory of the project and must list all libraries, programs or any kind of tool that your project needs to be built, such as a compiler, linker, libraries, etc.

Lazy people, like me, usually don't create it from scratch, I prefer to use the autoscan tool, that looks for things that you are using and generate a configure.scan file that can be used as the basis for your real configure.ac.

To inform automake that you will need python stuff, you can add this to your configure.ac:

dnl python checks (you can change the required python version bellow)
AM_PATH_PYTHON(2.7.0)
PY_PREFIX=`$PYTHON -c 'import sys ; print sys.prefix'`
PYTHON_LIBS="-lpython$PYTHON_VERSION"
PYTHON_CFLAGS="-I$PY_PREFIX/include/python$PYTHON_VERSION"
AC_SUBST([PYTHON_LIBS])
AC_SUBST([PYTHON_CFLAGS])

Wrap up

Basically, automake has a built-in extension that knows how to deal with python stuff, you just need to add it to your configure.ac file and then take the advantage of this feature in your Makefile.am.

PyGtk is definitely an awesome example, but it's pretty big, so maybe you will want to check another project, like Guake