Continuous Integration System for a Python Codebase

Question

I am starting to work on a hobby project with a Python codebase and I would like to set up some form of continuous integration (i.e. running a battery of test-cases each time a check-in is made and sending nag e-mails to responsible persons when the tests fail) similar to CruiseControl or TeamCity.

I realize I could do this with hooks in most VCSes, but that requires that the tests run on the same machine as the version control server, which isn't as elegant as I would like. Does anyone have any suggestions for a small, user-friendly, open-source continuous integration system suitable for a Python codebase?

Answer 1

We run Buildbot - Trac at work. I haven't used it too much since my codebase isn't part of the release cycle yet. But we run the tests on different environments (OSX/Linux/Win) and it sends emails — and it's written in Python.

Answer 2

One possibility is Hudson. It's written in Java, but there's integration with Python projects:

Hudson embraces Python

I've never tried it myself, however.

(Update, Sept. 2011: After a trademark dispute Hudson has been renamed to Jenkins.)

Answer 3

Second the Buildbot - Trac integration. You can find more information about the integration on the Buildbot website. At my previous job, we wrote and used the plugin they mention (tracbb). What the plugin does is rewriting all of the Buildbot urls so you can use Buildbot from within Trac. (http://example.com/tracbb).

The really nice thing about Buildbot is that the configuration is written in Python. You can integrate your own Python code directly to the configuration. It's also very easy to write your own BuildSteps to execute specific tasks.

We used BuildSteps to get the source from SVN, pull the dependencies, publish test results to WebDAV, etcetera.

I wrote an X10 interface so we could send signals with build results. When the build failed, we switched on a red lava lamp. When the build succeeded, a green lava lamp switched on. Good times :-)

Answer 4

We use both Buildbot and Hudson for Jython development. Both are useful, but have different strengths and weaknesses.

Buildbot's configuration is pure Python and quite simple once you get the hang of it (look at the epydoc-generated API docs for the most current info). Buildbot makes it easier to define non-testing tasks and distribute the testers. However, it really has no concept of individual tests, just textual, HTML, and summary output, so if you want to have multi-level browsable test output and so forth you'll have to build it yourself, or just use Hudson.

Hudson has terrific support for drilling down from overall results into test suites and individual tests; it also is great for comparing test output between builds, but the distributed (master/slave) stuff is comparatively more complicated because you need a Java environment on the slaves too; also, Hudson is less tolerant of flaky network links between the master and slaves.

So, to get the benefits of both tools, we run a single instance of Hudson, which catches the common test failures, then we do multi-platform regression with Buildbot.

Here are our instances:

• Jython Hudson
• Jython buildbot

Answer 5

We are using Bitten wich is integrated with trac. And it's python based.

Answer 6

TeamCity has some Python integration.

But TeamCity is:

• not open-source
• is not small, but rather feature rich
• is free for small-mid teams.

Answer 7

I have very good experiences with Travis-CI for smaller code bases. The main advantages are:

• setup is done in less than half a screen of config file
• you can do your own installation or just use the free hosted version
• semi-automatic setup for github repositories
• no account needed on website; login via github

Some limitations:

• Python is not supported as a first class language (as of time of writing; but you can use pip and apt-get to install python dependencies; see this tutorial)

• code has to be hosted on github (at least when using the official version)