Workflow for contributing to Python libraries under virtualenv

Question

Let's say that I want to contribute to thislib, which is a Python library that is available as a Git repository. Several applications, such as thislib_app utilize this library, and I want to contribute to the thislib and see how my optimizations work out for thislib_app (we can say that thislib_app is like one big unit test).

Here's my workflow:

1. Clone the thislib_app repo
2. Create a virtualenv called lib_env and install thislib
3. Run thislib_app. Oh no! Something breaks, but it's thislib's fault.
4. Go inside lib_env/local/lib/python/site-packages/thislib/x.py and make necessary changes
5. Test again. It works!

Now in order to make these changes known to maintainers, I must clone a fork of thislib's repository, copy OVER the changes I've made in the virtualenv to my development branch, and make a PR. If I decide to make more optimizations / changes, I would have to repeat this copy process over again.

What can I do to ensure an efficient workflow where I do not have to move around my changes and can get a PR up immediately?

Answer 1

The solution is pip's (actually just setuptools') development mode:

1. create a virtualenv for your app
2. git-clone the library repository
3. within the virtualenv, install the library using python setup.py develop or pip install -e . (the -e stands for editable)

Now, any changes to the library will be immediately live in the virtualenv, no extra copying needed.

You can commit and push the changes you made in the git repository.

The drawback is that this requires you to always have a working state of the library checked out while you want to use the app. There will also be restrictions if your setup.py does anything clever, e.g. if the library contains compiled extensions or if you are adding new entry_points.