Making Django go green

Question

I have a Django management command that makes thousands of TCP/UDP requests. I've used Gevent to speed this up as I've restructured my code to work as coroutines. The socket connections no longer block but from what I've read, parts of Django still aren't green. (By green, I mean using greenlets.)

Could you tell me what parts of Django aren't green and what I can do to make them green? There are some DB related parts that still block I think. Are there any libraries/patches for Django that help me make it green?

I'm not too concerned about the request/response cycle being green or not therefore, would Gunicorn help me?

Thanks

Answer 1

The gevent monkey patcher will patch the standard library to be Greenlet friendly. This should take of a lot of common Django calls.

from gevent import monkey; monkey.patch_all()

As far as databases, normally the interfaces are blocking. If you use PostgreSQL look into psyco_gevent to monkey patch psycopg2 to cooperate with gevent.

Answer 2

gevent will not green third-party libraries as long there are using either C extensions or using other third-party libraries which are using C extensions. If standard libraries are used gevent is able to monkey patch them.

Some database drivers rely of course on the database implementation itself. Just because you use a greened library does not mean you database will behave "green". This also need to be invoked on the database level itself. Prosgresql for example does support async command processing since version 2.2.

Also you might want to check Projects using gevent.

Gunicorn will make life easier for you, at least when you start to deploy your application. Also during development I would recommend gunicorn (with gevent) over the normal django runserver command, simply because it can handle simultaneous requests way faster.