Developing and using the same Python on the same computer

Question

I'm developing a Python utility module to help with file downloads, archives, etc. I have a project set up in a virtual environment along with my unit tests. When I want to use this module on the same computer (essentially as "Production"), I move the files to the mymodule directory in the ~/dev/modules/mymodule

I keep all 3rd-party modules under ~/dev/modules/contrib. This contrib path is on my PYTHONPATH, but mymodule is NOT because I've noticed that if mymodule is on my PYTHONPATH, my unit tests cannot distinguish between the "Development" version and the "Production" version. But now if I want to use this common utility module, I have to manually add it to the PYTHONPATH.

This works, but I'm sure there's a better, more automated way.

What is the best way to have a Development and Production module on the same computer? For instance, is there a way to set PYTHONPATH dynamically?

Answer 1

You can add/modify python paths at sys.path, just make sure that the first path is the current directory ".", because some third-party modules rely on importing from the directory of the current module.

More information on python paths: http://djangotricks.blogspot.com/2008/09/note-on-python-paths.html

Answer 2

I'm guessing by virtual environment you mean the virtualenv package?

http://pypi.python.org/pypi/virtualenv

What I'd try (and apologies if I've not understood the question right) is:

• Keep the source somewhere that isn't referenced by PYTHONPATH (e.g. ~/projects/myproject)
• Write a simple setuptools or distutils script for installing it (see Python distutils - does anyone know how to use it?)
• Use the virtualenv package to create a dev virtual environment with the --no-site-packages option - this way your "dev" version won't see any packages installed in the default python installation.
• (Also make sure your PYTHONPATH doesn't have any of your source directories)

Then, for testing:

• Activate dev virtual environment
• Run install script, (usually something like python setup.py build install). Your package ends up in /path/to/dev_virtualenv/lib/python2.x/site-packages/
• Test, break, fix, repeat

And, for production:

• Make sure dev virtualenv isn't activated
• Run install script
• All good to go, the "dev" version is hidden away in a virtual environment that production can't see...
• ...And there's no (direct) messing around with PYTHONPATH

That said, I write this with the confidence of someone who's not actually tried setting using virtualenv in anger and the hope I've vaguely understood your question... ;)

Answer 3

You could set the PYTHONPATH as a global environment variable pointing to your Production code, and then in any shell in which you want to use the Development code, change the PYTHONPATH to point to that code.

(Is that too simplistic? Have I missed something?)