Python/WSGI: Dynamically spin up/down server worker processes across installations

Question

The setup

Our setup is unique in the following ways:

• we have a large number of distinct Django installations on a single server.
• each of these has its own code base, and even runs as a separate linux user. (Currently implemented using Apache mod_wsgi, each installation configured with a small number of threads (2-5) behind a nginx proxy).
• each of these installations have a significant memory footprint (20 - 200 MB)
• these installations are "web apps" - they are not exposed to the general web, and will be used by a limited nr. of users (1 - 100).
• traffic is expected to be in (small) bursts per-installation. I.e. if a certain installation becomes used, a number of follow up requests are to be expected for that installation (but not others).

As each of these processes has the potential to rack up anywhere between 20 and 200 MB of memory, the total memory footprint of the Django processes is "too large". I.e. it quickly exceeds the available physical memory on the server, leading to extensive swapping.

I see 2 specific problems with the current setup:

• We're leaving the guessing of which installation needs to be in physical mememory to the OS. It would seem to me that we can do better. Specifically, an installation that currently gets more traffic would be better off with a larger number of ready workers. Also: installations that get no traffic for extensive amounts of time could even do with 0 ready workers as we can deal with the 1-2s for the initial request as long as follow-up requests are fast enough. A specific reason I think we can be "smarter than the OS": after a server restart on a slow day the server is much more responsive (difference is so great it can be observed w/ the naked eye). This would suggest to me that the overhead of presumably swapped processes is significant even if they have not currenlty activily serving requests for a full day.

• Some requests have larger memory needs than others. A process that has once dealt with one such a request has claimed the memory from the OS, but due to framentation will likely not be able to return it. It would be worthwhile to be able to retire memory-hogs. (Currenlty we simply have a retart-after-n-requests configured on Apache, but this is not specifically triggered after the fragmentation).

The question:

My idea for a solution would be to have the main server spin up/down workers per installation depending on the needs per installation in terms of traffic. Further niceties: * configure some general system constraints, i.e. once the server becomes busy be less generous in spinning up processes * restart memory hogs.

There are many python (WSGI) servers available. Which of them would (easily) allow for such a setup. And what are good pointers for that?

Answer 1

See if uWSGI works for you. I don't think there is something more flexible.

You can have it spawn and kill workers dynamically, set max memory usage etc. Or you might come with better ideas after reading their docs.