How to use virtualenvwrapper for ckan development?

Question

Using virtualenvwrapper to develop on the CKAN project is a bit more complex than the simple workflow that you would get from the virtualenvwrapper introduction, i.e. just doing mkvirtualenv ckan, following the CKAN source install instructions to install CKAN into your virtualenv, and then hacking away on CKAN.

If I understand the virtualenvwrapper docs correctly, it's better to use the mkproject ckan command so that you have a virtualenv called ckan hidden away in your WORKON_HOME (~/.virtualenvs by default) and a nice, tidy project dir called ckan in your PROJECT_HOME (~/Projects on my machine) that contains only the source code repos that you're actually working on, and isn't cluttered with all the other files that go into the virtualenv. But the project management docs for virtualenvwrapper aren't very detailed..

If I do mkproject ckan and then proceed to follow the ckan source install instructions, the pip install --ignore-installed -e git+https://github.com/okfn/ckan.git#egg=ckan command installs the ckan git repo at ~/.virtualenvs/ckan/src/ckan rather than in my ~/Projects/ckan dir where I want it. How to get the ckan git repo to be in my project dir?

I'm also going to want to hack on different CKAN extensions (some of which come in their own separate git repos) as part of the same virtualenvwrapper project, so I want virtualenvwrapper to install those in ~/Projects/ckan as well.

Answer 1

I'm answering my own question, the below seems to have worked well enough for me but I think this answer could be improved on by automating all the git cloning and pip installing etc. that I do manually...

The trick is to deviate from the ckan source install instructions slightly and git clone ckan into the project dir 'manually', then you need to do an additional python setup.py develop command to install ckan into your virtualenv (the pip install command would have handled this for you):

> mkproject ckan

This creates a ckan virtualenv at WORKON_HOME/ckan and a ckan project dir at PROJECT_HOME/ckan, activates the virtualenv, and cd's into the project dir. So far the virtualenv and project dir are 'empty', there's no ckan installed anywhere yet. Now clone the ckan git repo:

(ckan)~/Projects/ckan> git clone git@github.com:okfn/ckan.git

This will clone the git repo to ~/Projects/ckan/ckan. At this point ckan commands still won't work, you've cloned the git repo but you still haven't installed ckan into your virtualenv:

(ckan)~/Projects/ckan> cd ckan
(ckan)~/Projects/ckan/ckan> python setup.py develop

ckan commands still won't work until you also install ckan's dependencies into your virtualenv:

(ckan)~/Projects/ckan/ckan> pip install --ignore-installed -r requires/lucid_missing.txt -r requires/lucid_conflict.txt -r requires/lucid_present.txt
(ckan)~/Projects/ckan/ckan> pip install webob==1.0.8

Now, finally you can run ckan commands:

(ckan)~/Projects/ckan/ckan> paster make-config ckan development.ini
    (edit development.ini as necessary)
(ckan)~/Projects/ckan/ckan> paster serve development.ini
Starting server in PID 17356.
serving on 0.0.0.0:5000 view at http://127.0.0.1:5000

From now on, each time you want to start developing on ckan you can just run workon ckan to activate your ckan virtualenv and cd to your ckan project dir, then hack away.

If you also want to hack on a CKAN extension such as ckanext-harvest, you can install that git repo into your ckan project dir alongside the ckan git repo:

(ckan)~/Projects/ckan> git clone git@github.com:okfn/ckanext-harvest.git
(ckan)~/Projects/ckan> cd ckanext-harvest
(ckan)~/Projects/ckan/ckanext-harvest> python setup.py develop