Good tool for automatic setup and deployment of Django projects

Question

I'm starting out as an independent web developer and I recently scared away a client by giving a quote for a rather simple site that was quite a bit higher than they expected. It was basically brochureware that they could have done themselves in a hosted solution like Wordpress.com or Google sites. Except for one critical feature, that made me think that Django would be a nice fit, and that some proper web development could be motivated.

When looking at the tasks I've listed in my rough estimate it's pretty obvious to me that most of the hours in there are stuff that is not specific to this clients website. This got me thinking that I should have a script to automate the process of setting up and deploying new Django projects.

A couple of things would still have to be done manually like:

• Set up a new VPS (or in most cases just go with virtualhosts on an existing VPS).

• Edit DNS settings to point clientsdomain.com and test.clientsdomain.com to the new VPS.

• Set up SSH-keys

• Edit a config file.

The script would then do stuff like:

• Install some required packages on the new VPS

• Install default a iptables firewall on the new VPS

• Add nginx/apache config for clientsdomain.com and test.clientsdomain.com

• Set up new up databases for production and test on the VPS and for development on my local machine.

• Create a new Django project and put it on Bitbucket.

• Set up settings.py and local_settings.py for production, test and development

• Set up offsite backuproutines for production database and uploaded files directory.

• Enable some default apps: flatpages, admin

• Add some boilerplate templates, a couple of boilerplate flatpages and a navbar.

• Add the 960.gs CSS-framework

• Add jQuery

• Set up djapian or Haystack (including cron job for automatic updating) for search.

• Set up a new Google analytics profile.

• Include possibility to replicate databases between production, test and development

• Set up Pingdom monitoring

I've used Rake and Fabric for somewhat similar (but less ambitious) stuff in the past and I'm thinking that Fabric might be a pretty good fit for this task as well, but I'd still like your input. Are there other tools I should look into? I've heard good things about Puppet but just looking at their site (it contains the word Enterprise ) gives me the feeling that it might be overkill for a one man operation.

Answer 1

We currently do it with fabric+buildout. Others say Chef or Puppet is better suited (and it probably is, if you want to do server level stuff, not only app-level).

Also for Django there's a few dedicated hosters which take a lot of load off you, I especially like http://ep.io which we use to power our fully automated deployements for the demos of the django-cms, so maybe you should look into those hosting providers too rather than limiting yourself to VPSs which are more overhead for you.

Answer 2

Puppet may seem daunting and overkill for small projects since it's so often used for huge deployments, but I use it to manage just one machine in standalone mode without a client server setup so that I don't have to deal with SSL certs and multiple machines, which keeps things a lot simpler, but still gives me the benefit that I can do really fast disaster recovery or move my hosting without a lot of effort. There's some great reasons (idempotency, cross platform support, full lifecycle management, abstraction, concise DSLs) for using modern configuration management systems over systems that are essentially scripts that do ssh in a loop or relying on platforms that lock you in.

Check out learning puppet for a quick ramp up including examples and a VM playground. You can get really useful things done with simple Puppet scripts (manifests) that run standalone, and then start learning all the advanced features once you need them.

Another nice thing is that a lot of Puppet manifests and modules have already been written by others, and they're shared on the Puppet Forge and by many other advanced Puppet users.

Answer 3

I've been considering a lot of this recently as someone in a similar position.

A lot of what you have mentioned can be helped along by setting up some good skeleton code that can be pulled from git for every new project.

Have a look at this for a good starting point for a generic django project skeleton code

http://blog.zacharyvoase.com/2010/02/03/django-project-conventions/

I put together something similar that allows me to get up and running quickly, but also to separate the server stuff from the project stuff. This is very important as it allows you to version control every project without including system/server files. It's here (still very much in progress). This takes care of folder layout, extra css, boilerplate html stuff, grid/960 stuff, jquery, development vs production settings, database settings (mostely), default installed apps etc. Here is the layout explained

Using Virtualenv, Virtualenvwrapper & pip allows you to set up standalone, encapsulated python enviroments which are great for running multiple projects on one VPS. Pip allows you to install packages to a particular virualenv and also to output all your packages to a textfile that can be imported later on. This makes redeploying code from development to production very quick. It also allows you to write a generic Requirements file in your skeleton code that automatically installs all your normal django apps i.e. django-tagging etc.

In terms of databases, I've stopped trying to have development databases and production databases on different machines, it's too hard to import/export fixtures. Now I just have separate production & dev databases on the VPS and connect remotely (over ssh) to the dev one when developing. You can easily copy one to another which is nice also.

When everything is ready then, you can deploy from development to production using fabric (i'm yet to get stuck into this so I'm not sure of its ease of use)

I'd be very interested to hear other peoples thoughts about this as I was about to post something simliar!

Answer 4

Check out Silk Deployment, which is based on Fabric and is used to deploy Django (or any WSGI) + Nginx + Gunicorn:

http://pypi.python.org/pypi/silk-deployment/