Install package dependencies on CI-server virtual env

Question

For my application environment I use virtualenv. I have a script which installs the virtualenv and installs all needed packages based on requirements.txt.

This works perfectly for our distribution in our corporate network. I use a network share where all needed packages are placed. Each production system has access to it an can install it from there. My local dev-environment uses a local folder where all the packages located.

Now the problem: I want to setup a build server for continuous integration. Unfortunately, this server does not have access to the network-share but it has access to the packages via git.

What's the best way to also let the build-server install its dependencies from requirements.txt?

Answer 1

If your build server can access the packages via git and you want to install using requirements.txt, you're in luck.

Installing Python Packages with git via requirements.txt

You can specify git based packages in requirements.txt. Just put them on separate lines as if they were another package:

git+git://github.com/thadeusb/flask-cache.git

You can even pin to a specific commit using @:

git+git://github.com/thadeusb/flask-cache.git@40cfd9280dc66ea54df0961420fc94853d506a35

Bonus Mode

If you want to pull from a repository in editable mode, prepend a -e:

-e git+git://github.com/thadeusb/flask-cache.git@40cfd9280dc66ea54df0961420fc94853d506a35#egg=Flask-Cache

Roll your own PyPI Server

Though you didn't ask for it specifically, you could also roll your own Python Package Index.

Once you have it configured, you simply have to set the index-url to fetch packages from. You can do this directly:

pip install --index-url=http://pip.razer.domain/ -r requirements.txt

Or set up a config file ~/.pip/pip.conf:

[global]
index-url = http://pip.razer.domain/