Chef workflow for new cookbooks

Question

While building my own cookbooks I find myself constantly within this cycle:

1. Change cookbook on my local computer
2. Upload modified cookbook to chef server
3. Run chef-client on remote machine
4. Repeat

Since I am new to chef, I repeat that cycle extremely often, but I find rather cumbersome uploading and downloading so frequently from the chef server.

How experienced chef users ease this cycle? I learnt Chef with the free hosted solution, but I am not sure if I should be better off using just Chef-solo and move back later to the hosted version once I have many servers and more experience with chef.

Is there maybe a workflow where I can quickly try changes to my cookbooks directly on my remote machine while using hosted Chef but without uploading them?

Answer 1

I do all my development locally on my laptop, using Vagrant. When the cookbook is ready, it gets pushed to it's new Git repository home and integrated into my production chef server infrastructure.

The following example runs nginx on a virtualbox image.

Example

Install vagrant plugins (only needs to be done once):

vagrant plugin install vagrant-omnibus
vagrant plugin install vagrant-berkshelf
vagrant plugin install vagrant-chef-zero

Use Berkshelf to generate a new application cookbook.

$ berks cookbook demo
      create  demo/files/default
      create  demo/templates/default
      create  demo/attributes
      create  demo/definitions
      create  demo/libraries
      create  demo/providers
      create  demo/recipes
      create  demo/resources
      create  demo/recipes/default.rb
      create  demo/metadata.rb
      create  demo/LICENSE
      create  demo/README.md
      create  demo/Berksfile
      create  demo/Thorfile
      create  demo/chefignore
      create  demo/.gitignore
         run  git init from "./demo"
      create  demo/Gemfile
      create  demo/Vagrantfile

Edit the following files (Details below):

1. Vagrantfile <-- Controls vagrant's operation
2. metadata.rb <-- List community cookbook dependencies
3. recipes/default.rb <-- Calls to community cookbook recipes and LWRPs

Running vagrant will startup a virtual machine that is provisioned using chef-client. The chef zero plugin will run a local embedded instance of chef-server. The berkself plugin is used to automatically load the cookbook dependencies.

vagrant up

The following command will rerun chef-client (following edits):

vagrant provision

Finally, the really big advantage of doing development, using local virtualization, is that you can tear everything down and build it again from scratch:

vagrant destroy -f && vagrant up

Vagrantfile

Controls vagrant's operation. In this case I'm only starting a single VM, provisioned using chef-client:

Vagrant.require_plugin "vagrant-omnibus"
Vagrant.require_plugin "vagrant-berkshelf"
Vagrant.require_plugin "vagrant-chef-zero"

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  # Box config
  config.vm.box = "saucy64"
  config.vm.box_url = "http://cloud-images.ubuntu.com/vagrant/saucy/current/saucy-server-cloudimg-amd64-vagrant-disk1.box"

  # Virtualbox config
  config.vm.provider :virtualbox do |vb|
    vb.customize ["modifyvm", :id, "--memory", 1024]
  end

  # Networking config
  config.vm.network "private_network", ip: "10.0.0.10"

  # Plugin config
  config.omnibus.chef_version = :latest
  config.chef_zero.enabled = true
  config.berkshelf.enabled = true

  # Chef config
  config.vm.provision :chef_client do |chef|
    chef.add_recipe "demo"
  end
end

The omnibus plugin is responsible for installing the desired version of chef. The Berkshelf plugin will download cookbook dependencies and when combined with chef-zero will upload cookbooks during each provision run.

metadata.rb

Add nginx as a cookbook dependency:

name             'demo'
maintainer       'YOUR_NAME'
maintainer_email 'YOUR_EMAIL'
license          'All rights reserved'
description      'Installs/Configures demo'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version          '0.1.0'

depends "nginx"

The Berksfile is preconfigured to load the cookbook dependencies, listed in the metadata file, from the community cookbook repository.

site :opscode

metadata

recipes/default.rb

Run the default nginx recipe

#
# Cookbook Name:: demo
# Recipe:: default
#
# Copyright (C) 2014 YOUR_NAME
# 
# All rights reserved - Do Not Redistribute
#

include_recipe "nginx"

Answer 2

I go through that loop most of the time during my testings. But I also use Chef solo when I need to try several minor changes in a recipe. If you have a test machine with a local copy of your chef repo then you can try making changes there and running chef solo to see the results. This way there's no uploading.