How can I get Chef to run apt-get update before running other recipes

Question

Right now I have the following in my Vagrantfile:

config.vm.provision :chef_solo do |chef|
    chef.cookbooks_path = "cookbooks"
    chef.add_recipe "apt"
    chef.add_recipe "build-essential"
    chef.add_recipe "chef-redis::source"
    chef.add_recipe "openssl"
    chef.add_recipe "git"
    chef.add_recipe "postgresql::server"
    chef.add_recipe "postgresql::client"
end

In order to install the software added to my recipe_list, I need to get the VM to issue an apt-get update before installing the other software.

I was under the impression that this was one of the features of the 'apt' recipe - that it would run the update first thing.

The output when I do a vagrant provision is:

[Sat, 11 Feb 2012 22:20:03 -0800] INFO: *** Chef 0.10.2 ***
[Sat, 11 Feb 2012 22:20:03 -0800] INFO: Setting the run_list to ["recipe[apt]", "recipe[build-essential]", "recipe[chef-redis::source]", "recipe[openssl]", "recipe[git]", "recipe[postgresql::server]", "recipe[postgresql::client]", "recipe[vagrant-main]"] from JSON
[Sat, 11 Feb 2012 22:20:03 -0800] INFO: Run List is [recipe[apt], recipe[build-essential], recipe[chef-redis::source], recipe[openssl], recipe[git], recipe[postgresql::server], recipe[postgresql::client], recipe[vagrant-main]]
[Sat, 11 Feb 2012 22:20:03 -0800] INFO: Run List expands to [apt, build-essential, chef-redis::source, openssl, git, postgresql::server, postgresql::client, vagrant-main]
[Sat, 11 Feb 2012 22:20:03 -0800] INFO: Starting Chef Run for lucid32
[Sat, 11 Feb 2012 22:20:03 -0800] INFO: Processing package[postgresql-client] action install (postgresql::client line 37)
[Sat, 11 Feb 2012 22:20:04 -0800] ERROR: package[postgresql-client] (postgresql::client line 37) has had an error
[Sat, 11 Feb 2012 22:20:04 -0800] ERROR: Running exception handlers
[Sat, 11 Feb 2012 22:20:04 -0800] ERROR: Exception handlers complete
[Sat, 11 Feb 2012 22:20:04 -0800] FATAL: Stacktrace dumped to /tmp/vagrant-chef-1/chef-stacktrace.out
[Sat, 11 Feb 2012 22:20:04 -0800] FATAL: Chef::Exceptions::Exec: package[postgresql-client] (postgresql::client line 37) had an error: apt-get -q -y install postgresql-client=8.4.8-0ubuntu0.10.04 returned 100, expected 0

Answer 1

I seem to have been able to solve the issue by applying the following patch:

https://github.com/wil/cookbooks/commit/a470a4f68602ec3bf3374830f4990a7e19e9de81

Answer 2

You can include the apt recipe in the very beginning:

include_recipe 'apt'

this will run the update command.

Answer 3

apt-get update should be running first the way you have it. However, the recipe will only update once every 24 hours:

execute "apt-get-update-periodic" do
  command "apt-get update"
  ignore_failure true
  only_if do
    File.exists?('/var/lib/apt/periodic/update-success-stamp') &&
    File.mtime('/var/lib/apt/periodic/update-success-stamp') < Time.now - 86400
  end
end

Answer 4

There are three resources that will do this nicely on an ubuntu system, specifically in use on 12.04 precise 64 bit by me.

1. run apt-get-update at will when other recipes require

2. install update-notifier-common package that gives us timestamps on updates

3. check the timestamps and update periodically. In this case below after 86400 seconds.

And here's those three recipes.

execute "apt-get-update" do
  command "apt-get update"
  ignore_failure true
  action :nothing
end

package "update-notifier-common" do
  notifies :run, resources(:execute => "apt-get-update"), :immediately
end

execute "apt-get-update-periodic" do
  command "apt-get update"
  ignore_failure true
  only_if do
   File.exists?('/var/lib/apt/periodic/update-success-stamp') &&
   File.mtime('/var/lib/apt/periodic/update-success-stamp') < Time.now - 86400
  end
end

Answer 5

It looks like the latest version of the opscode apt cookbook allow you to run it at compile time.

config.vm.provision :chef_solo do |chef|
  chef.cookbooks_path = "cookbooks"
  chef.add_recipe "apt"
  chef.json = { "apt" => {"compiletime" => true} }
end

As long as apt is run before other compiletime cookbooks (like postgres) in the run list, this should work.

Answer 6

A lot of the other answers posted here are likely to cause warnings about resource cloning.

According to the Apt cookbook documentation, you're supposed to be able to make this happen by setting node['apt']['compile_time_update'] = true, however I have never had much luck with this approach myself.

Here's what I do instead:

This will load the original apt-get update resource and ensure that it runs without adding a duplicate entry to the resource collection. This will cause apt-get update to execute during every Chef run during the compile phase:

# First include the apt::default recipe (so that `apt-get update` is added to the collection)
include_recipe 'apt'

# Then load the `apt-get update` resource from the collection and run it
resources(execute: 'apt-get update').run_action(:run)

Obviously, you'll also want to include the apt cookbook in your metadata.rb file:

# ./metadata.rb
depends 'apt'

Answer 7

The simplest and most direct way to solve the problem is by applying the following patch (h/t @ashchristopher):

https://github.com/wil/cookbooks/commit/a470a4f68602ec3bf3374830f4990a7e19e9de81

The problem is that the postgresql::client recipe runs the install action on the package resources at postgresql/recipes/client.rb:39 and 44 at compile-time rather than run-time like normal (h/t Tim Potter), causing them to be evaluated by Chef (and thus installed) before anything else runs.

pg_packages.each do |pg_pack|
  package pg_pack do
    action :nothing
  end.run_action(:install)
end

gem_package "pg" do
  action :nothing
end.run_action(:install)

I believe this is done in service of the database cookbook's postgres provider, which depends on the postgresql cookbook and relies on the pg gem being installed before it will compile. Applying the above patch may break the database cookbook.

The other alternative solution would be to create a recipe which runs apt-get update also at compile time and put it in your run_list before the postgresql cookbook. In its simplest form that would probably be something like:

execute "apt-get update" do
  action :nothing
end.run_action(:install)

Answer 8

Without patching, this is a generic approach to the problem that will update on every run:

bash "update-apt-repository" do
  user "root"
  code <<-EOH
  apt-get update
  EOH
end

It may be worth considering that such a run, on every run, ties up a fair bit of system resources for about 30 seconds; you may want to have a special recipe named recipe::update_apt that you have run via cron or via some other event i.e.

chef-client -o "recipe[yourrecipe::update_apt]"

Answer 9

To run apt-get update at compile time, do:

e = execute "apt-get update" do
  action :nothing
end

e.run_action(:run)

check https://wiki.opscode.com/display/chef/Evaluate+and+Run+Resources+at+Compile+Time

Answer 10

I had the same situation, and in my case, apt cookbook was second after the one which called installation of package. Just leaving it here so maybe someone will benefit from it. Check the order of cookbooks in your runlist, role or wherever else.

Answer 11

just a friendly reminder that adding all those recipes inside the vagrant provision can quickly become unmanageable.

A better pattern is to create a chef role chef/my-fancy-role.rb

# Name of the role should match the name of the file
name "my-fancy-role"

# Run list function we mentioned earlier
run_list(
    "recipe[apt]",
    "recipe[build-essential]",
    "recipe[chef-redis::source]",
    "recipe[openssl]"
)

And then add this role to the Vagrantfile provisioning section:

config.vm.provision :chef_solo do |chef|
  chef.roles_path = "chef/roles"
  chef.cookbooks_path = ["chef/site-cookbooks", "chef/cookbooks"]
  chef.add_role "my-fancy-role"
end

Answer 12

For the recent Chef version, i.e version 14. You could also use https://docs.chef.io/resource_apt_update.html

apt_update

The below output was my experiment running that resource for chef_14.5.33 in local mode (zero):

curl -O https://packages.chef.io/files/stable/chef/14.5.33/ubuntu/18.04/chef_14.5.33-1_amd64.de
sudo dpkg -i chef_14.5.33-1_amd64.deb
mkdir -p cookbooks/hello/recipes/ && echo "apt_update" > cookbooks/hello/recipes/default.rb

sudo sh -c 'chef-client -z -o hello'
[2018-10-12T10:25:30+00:00] WARN: No config file found or specified on command line, using command line options.
Starting Chef Client, version 14.5.33
[2018-10-12T10:25:32+00:00] WARN: Run List override has been provided.
[2018-10-12T10:25:32+00:00] WARN: Run List override has been provided.
[2018-10-12T10:25:32+00:00] WARN: Original Run List: []
[2018-10-12T10:25:32+00:00] WARN: Original Run List: []
[2018-10-12T10:25:32+00:00] WARN: Overridden Run List: [recipe[hello]]
[2018-10-12T10:25:32+00:00] WARN: Overridden Run List: [recipe[hello]]
resolving cookbooks for run list: ["hello"]
Synchronizing Cookbooks:
  - hello (0.0.0)
Installing Cookbook Gems:
Compiling Cookbooks...
Converging 1 resources
Recipe: hello::default
  * apt_update[] action periodic (up to date)
[2018-10-12T10:25:32+00:00] WARN: Skipping final node save because override_runlist was given
[2018-10-12T10:25:32+00:00] WARN: Skipping final node save because override_runlist was given

Running handlers:
Running handlers complete
Chef Client finished, 0/1 resources updated in 01 seconds