What are the pros and cons of running (i) gunicorn system-wide and (ii) a gunicorn install per virtualenv (assuming 5-6 virtualenvs per 512mb VM)

Question

I am running 5-6 websites on top of django. Each django project is in its own virtualenv, and serves a single website.

I currently install gunicorn per virtualenv, and thus each django project has its own gunicorn installation and processes.

What are the pros and cons of this approach, and the alternative of installing gunicorn system wide, with one gunicorn installation managing all the websites (like a standard apache setup).

The environment is a 512mb VM running Ubuntu 11.04 on linode. Of course, I use nginx to proxy requests to gunicorn.

Answer 1

On a bunch of our servers we have as many as 10 different django applications all running their own gunicorn, in their own virtualenv.

We use one supervisor process system wide to control them all.

We also have one nginx process that handles all traffic (reverse proxy, serving static media, etc) for all of the applications.

We decided to use this approach because it seems the most sane and easiest to setup and control each application on it's own. Using supervisor to start/stop/reload each app on their own, gives us more fine grain control over what we want to do. It also makes it really easy to add more applications, and not affect the ones that are already on the machine.

It also allows us to run each application as a different user, so that we have better control of what those processes are doing, and what they have access too, which is good if you need to worry about security. If these are all your own apps on your own server under your desk, then this might not be an issue for you.

If you run one version of gunicorn for everything, and anything happened to that gunicorn process then it will affect all of your applications, which isn't ideal. It also limits what you can do, if you want to run eventlet on one application, gevent on another and sync on another, it is easier if they are all running separately. Also some applications might need more processes then others, and it is easier to configure this if they are separate.

If you do the apache approach, it might reduce your overall resource footprint because you won't have to have multiple versions of the same processes running for each application. If you're resource constraint, this might be a bigger deal for you, but nowadays memory and CPU is so cheap, that it isn't an issue.

A lot of this really depends on how much traffic each of these applications are going to be getting. If you aren't getting any traffic at all, then do what is easiest for you to setup and maintain. If you are getting a ton of traffic, or plan on getting a ton, then what ever you have setup now, is probably going to have to change again once you outgrow your current setup.

Answer 2

As I see it, you’re talking about three options:

• One single Gunicorn process for all your projects
• One single Gunicorn installation, with separate processes for each project
• Multiple Gunicorn installations, one in each virtualenv

I’m not sure whether the first is even possible, but the Gunicorn docs all seem geared to one of the other two options, and I think it usually needs to be spawned with a single WSGI module. Besides, you’d also lose the flexibility of being able to kill or restart the servers for individual projects without disturbing the others.

As for whether to install a separate Gunicorn for each virtualenv, this is certainly the approach suggested by the Gunicorn team, and the one that would give you the flexibility to use features from a newer version of Gunicorn on a new project without potentially breaking existing versions.

Short answer: you need to have separate Gunicorn instances for each project; whether you have separate installations as well is up to you, but it’ll probably save you time and headaches.