common sense advice about python package management on OSX for beginners

StackOverflow https://stackoverflow.com/questions/4835899

Question

If I'm learning python development on OSX (Snow Leopard), how should I best manage my python packages?

I am very excited about the large number of great python packages out there. I would love to start learning them, but package management has got me totally confused. Clearly I have no idea what I'm doing. I see many questions on Stack Overflow that address package management, and people frequently answer them by saying "Why don't you just use x? Look how easy it is in this single line I copied from my shell,"

$ x install something

Which is a great answer until x fails to handle some specific dependencies or can't handle package y. Here's what I've already attempted:

  1. I started out just installing things with easy_install.
  2. I soon found that some important packages had missing dependencies (PIL) so I hunted around and discovered MacPorts.
  3. I started using MacPorts and all my environment variables got really confusing, so I learned enough bash to reconfigure PATH, PYTHONPATH, and sys.path to get things working (and to make my terminal window look sweet). MacPorts seemed great for a while.
  4. But then I got confused about which version of different modules I was using, and I found more dependency issues (GDAL) ...
  5. Then several very intelligent people strongly recommended I switch to Homebrew. So I uninstalled MacPorts and installed Homebrew.
  6. Now that I have Homebrew, I can't seem to install much of anything, because I don't know how to create Formula for most of the python packages I want to use. If a package is listed at pypi, I don't know if I should actually use Homebrew, or just figure out how to configure easy_install to install to Homebrew directories. And I still don't know how I should best approach missing dependencies. Maybe there's some secret repository of python-related Formula? or an easy way of converting portfiles into Formula?

I know that the right answer might depend on what I'm trying to install. I'm trying to install sets of packages that let me:

  1. play around and automate everything with python
    • appscript
    • PIL
    • pygame
  2. do GIS related work in python
    • gdal
    • shapely
    • postgis
    • scipy
  3. explore the wondrous possibilities of web apps
    • django
    • google-app-engine

I'm fully willing to wipe everything off my machine, which must be a bit cluttered with scraps of packages by now. If I should be learning virtualenv, or just learning more about the basic process of checking dependencies and installing them myself, I'm happy to do that. If I should just dig into Homebrew, I happy to do that as well. How should I deal with these issues, which often prevent me from doing what I really want to do (make neat stuff with python)?

Was it helpful?

Solution

In my opinion, the best way to handle python packages is to have a set of Python installations separate from the system Python, one for each version you need. This way I do not pollute the main python with old versions of packages, and I can delete them and reinstall them without any of my projects having problems.

Details:

On OS X, using the Macports versions are fine. The same is probably true for homebrew, although I never used it. (On Linuxes I install Python from source into /opt/pythonxx/, for example /opt/python25/, /opt/python26/, /opt/python27 and /opt/python31.)

I then never install any packages into these Pythons, with the exception of virtualenv, distribute and pip, which I install in all of them. When I want to install a package for a project, I make a virtualenv for that project with the python version I use for it:

$ cd /home/projects
$ /opt/python27/bin/virtualenv acoolproject
$ cd acoolproject
$ ./bin/pip install ThepackageIneed.

As you install each projects dependencies separately, you avoid dependency issues and version conflicts and version confusion.

You can enable and disable environments with virtualenv, I tend never to do that, I use the python executable explicitly instead:

$ ./bin/python main.py

or 

$ /home7projects/acoolproject/bin/python

If I need things that you can't just install with pip, like Plone, or nginx or varnish, I use zc.buildout to make a replicatable environment configuration, zc.buildout will also run in isolation, so no packages gets installed in the main pythons.

OTHER TIPS

If you're using Homebrew, you'll probably want to use pip to install Python packages. pip can be installed via Homebrew, but Homebrew doesn't include formulae for other Python packages. But it's pretty easy to install packages using pip -- as simple as

$ pip install <package>

(The package name can be found on PyPI.) If you installed pip via Homebrew, that's all you need to do.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top