Have a look at buildout. From their site:
Buildout is a Python-based build system for creating, assembling and deploying applications from multiple parts, some of which may be non-Python-based. It lets you create a buildout configuration and reproduce the same software later.
We are using it to do exactly what you describe. It is flexible and easy to extend through recipes.