Puppet in the cloud?

Question

Is it possible to have a Puppet setup where you use JClouds to instantiate new virtual machines on your cloud, but then have their configuration (software stack) defined and implemented through Puppet?

Or is there something inherent to the nature of Puppet that prevents its use on a cloud provider like AWS, RackSpace or Heroku?

Answer 1

Puppet is an excellent choice for configuring your cloud infrastructure, most cloud providers allow you to call a script on first boot (Ec2 has user-data), you can make this script insert some node type data then clone a puppet repo and apply it, if you don't want to run a puppet master service (which can be a hassle to setup and maintain), you can also use git to push updates to the configuration and even generate a new image on config changes to allow rapid node launches with you latest setup. Check out this blog

Answer 2

There are two separate issues involved here: bootstrapping puppet on cloud nodes and orchestrating between them (e.g. configure application servers with the ip address of the database).

For bootstrapping, there are many tools available; AWS CloudFormation can be integrated using user-data, CloudInit (default on Ubuntu, ec2-linux AMIs and on many EL images) supports bootstrapping puppet out of the box. Puppetlabs offer cloud provisioner, and lastly, there is Cloudify. Other then CloudFormation and Cloudify, most tools don't manage your stack after bootstrapping and do not offer orchestration. CloudFormation itself offers only boot time orchestration and it is pretty lame. Puppet itself is lacking with respect to orchestration (compared with Chef's excellent search feature for example)

Cloudify provides ongoing stack management and fancy orchestration exposed via Puppet integration module. This gives you the ability to pass information between nodes (for service discovery, credential distribution, etc.) and bootstrap your entire system with one command. Plus it supports most clouds in existence.

Answer 3

The only open source tool I'm aware of that directly announces puppet + jclouds integration is Apache Whirr, but this is mainly via puppet apply vs a server.

As puppet has a restful interface now, it should be possible to create native puppet support in jclouds as we do for chef. You'd be welcome to help define that :)

In the mean time, you can either use something like whirr, or craft instructions and supply them as a shell script via jclouds TemplateOptions.runScript during node creation, or later using the submitScriptOnNode command (both of which currently operate via ssh).