Multiple installations of my app - how do I handle it

Question

I have an app written in PHP, MySQL, etc. The app has a few dependencies such as beanstalkd, Solr and a few PHP extensions.

For each customer we have a separate installation of the app, either on a server shared with other customers or on a server with only that customer.

For now we're using a Puppet script to bootstrap new customers and then we manually go to each customer to make a git pull, update the db, etc., whenever something changes.

What we're looking for is really a tool that has as many of the following features as possible:

1. Web interface that allows us to see all customers and their current revision
2. Ability to bootstrap new installations
3. Ability to update existing installations to a specific revision or branch

We're not looking for a tool to bootstrap new servers - we still do that manually. Instead we're looking for a way to automate the setup of clients on an existing server.

Would Chef or Puppet be sufficient for this, is there a more suitable tool, or would you recommend rolling something ourselves?

Answer 1

I'm a full time developer working on Puppet at Puppet Labs. I'm also the co-author of Pro Puppet.

Puppet is certainly sufficient for your goals. Here's one way to solve this problem using Puppet. First, I'll address the dependency management since these should only be managed once regardless of how many instances of the application are being managed. Then, I'll address how to handle multiple installations of your app using a defined resource type in Puppet and the vcsrepo resource type.

First, regarding the organization of the puppet code to handle multiple installations of the same app. The dependencies you mention such as beanstalkd, solr, and the PHP extensions should be modeled using a Puppet class. This class will be included in the configuration catalog only once, regardless of how many copies of the application are managed on the node. An example of this class might be something like:

# <modulepath>/site/manifests/app_dependencies.pp
class site::app_dependencies {
  # Make all package resources in this class default to
  # being managed as installed on the node
  Package { ensure => installed }
  # Now manage the dependencies
  package { 'php': }
  package { 'solr': }
  package { 'beanstalk': }
  # The beanstalk worker queue service needs to be running
  service { 'beanstalkd':
    ensure  => running,
    require => Package['beanstalk'],
  }
}

Now that you have your dependencies in a class, you can simply include this class on the nodes where your application will be deployed. This usually happens in the site.pp file or in the Puppet Dashboard if you're using the web interface.

# $(puppet config print confdir)/manifests/site.pp
node www01 {
  include site::app_dependencies
}

Next, you need a way to declare multiple instances of the application on the system. Unfortunately, there's not an easy way to do this from a web interface right now but it is possible using Puppet manifests and a defined resource type. This solution uses the vcsrepo resource to manage the git repository checkout for the application.

# <modulepath>/myapp/manifests/instance.pp
define myapp::instance($git_rev='master') {
  # Resource defaults.  The owner and group might be the web
  # service account instead of the root account.
  File {
    owner => 0,
    group => 0,
    mode  => 0644,
  }
  # Create a directory for the app.  The resource title will be copied
  # into the $name variable when this resource is declared in Puppet
  file { "/var/lib/myapp/${name}":
    ensure => directory
  }
  # Check out the GIT repository at a specific version
  vcsrepo { "/var/lib/myapp/${name}/working_copy":
    ensure   => present,
    provider => git,
    source   => 'git://github.com/puppetlabs/facter.git',
    revision => $git_rev,
  }
}

With this defined resource type, you can declare multiple installations of your application like so:

# $(puppet config print confdir)/manifests/site.pp
node www01 {
  include site::app_dependencies
  # Our app instances always need their dependencies to be managed first.
  Myapp::Instance { require => Class['site::app_dependencies'] }

  # Multiple instances of the application
  myapp::instance { 'jeff.acme.com': git_rev => 'tags/1.0.0' }
  myapp::instance { 'josh.acme.com': git_rev => 'tags/1.0.2' }
  myapp::instance { 'luke.acme.com': git_rev => 'tags/1.1.0' }
  myapp::instance { 'teyo.acme.com': git_rev => 'master' }
}

Unfortunately, there's not currently an easy to use out of the box way to make this information visible from a web GUI. It is certainly possible to do, however, using the External Node Classifier API. For more information about pulling external data into Puppet please see these resources:

• External Nodes Documentation
• R.I. Pienaar's Hiera (Hierarchical data store)

Hope this information helps.