Minimize service restarts from chef notifications?

Question

Currently my recipe with attributes that matter is structured like this:

service 'myservice' do
  action :nothing
  supports :status => true, :start => true, :stop => true, :restart => true
end

package 'packagename' do
  ...
end

template 'configfile1'
  notifies :restart, 'service[myservice]'
end
...
template 'configfileN'
  notifies :restart, 'service[myservice]'
end

execute "a command from package which generates and enables the init script" do
  notifies :start, 'service[myservice]', :immediately
end

execute "a command that should run once every time, that requires service to be running"

By doing this, we ensure that the initial start of the service has the config files, during every run the service is running for the second execute block, and if any config file changes, we restart the service to pick up the changes.

However, If a chef run occurs where the initial state of the service is stopped, (such as on the first run or if something bad happened), and config files have changed (notably on the first run, but possible for other runs), the first execute block will cause the service to start with the correct configuration files already in place, then at the end of the run, the service will restart unnecessarily. (Assume of course that resources after the initial start do not cause a restart of the service)

Changing the target action of notifications doesn't seem to work (as immediate notifications will still happen immediately, then delayed notifications still happen), and besides wouldn't be correct.

Also we can't subscribe the 2nd execute to the service start since if it's already running, we would wind up not executing it.

It is very nit picky, but is there a better pattern that could be followed to minimize restarts of a service for the initial run? Or a mechanism to cancel delayed notifications when a particular action is taken?

Answer 1

I used a flag file to achieve this.

Like below "apt-get update" is not executed once.

cookbook_file '/etc/apt/sources.list.d/maven.list' do
  source "repo_files/ubuntu_maven.list"
  cookbook "fluig-files"
  mode 0644
  notifies :run, "execute[should_update_repo]", :immediately
end

cookbook_file '/etc/apt/sources.list.d/couchbase.list' do
  source "repo_files/ubuntu_couchbase.list"
  cookbook "fluig-files"
  mode 0644
  notifies :run, "execute[should_update_repo]", :immediately
end

execute "apt-key couchbase" do
  command "apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A3FAA648D9223EDA"
  action :run
  not_if "apt-key list | grep couchbase"
  notifies :run, "execute[should_update_repo]", :immediately
end

execute "should_update_repo" do
  command "touch /tmp/should_update_repo"
  action :nothing
end

# Update system
execute "apt-get update" do
  command "rm -rf /tmp/should_update_repo && apt-get update"
  action :run
  only_if "test -f /tmp/should_update_repo"
end

Answer 2

Chef is designed to express configuration policy (system state), which is a slightly different thing than expressing a sequence of tasks to perform.

Fortunately since the DSL is ruby-based, resources can be redefined at runtime.

template "configfile1" do
  notifies :create, "ruby_block[restart_service1]", :immediately
end

template "configfile2" do
  notifies :create, "ruby_block[restart_service1]", :immediately
end

service "service1" do
  action [:enable, :start]
end

ruby_block "restart_service1" do
  block do

    r = resources(:service => "service1")
    a = Array.new(r.action)

    a << :restart unless a.include?(:restart)
    a.delete(:start) if a.include?(:restart)

    r.action(a)

  end
  action :nothing
end

This will rewrite the service resource to have "action [:enable, :restart]" instead of "action [:enable, :start]", if any of the template resources change state this run. So the ordering stays the same, and you only get one call through the service resource, instead of potentially three.

Answer 3

I am not sure, if will solve your problems with restarting the service too many times, but this structure seems more logical to me.

#Prepare everything to start the service
package 'packagename' do
  ...
end

template 'configfile1'
  notifies :restart, 'service[myservice]'
end
...
template 'configfileN'
  notifies :restart, 'service[myservice]'
end

execute "a command from package which generates and enables the init script" do
  notifies :restart, 'service[myservice]'
end

#Start the service
service 'myservice' do
  action :start
  supports :status => true, :start => true, :stop => true, :restart => true
end

#At this point service is surely running
execute "a command that should run once every time, that requires service to be running"

Every resource that changes the configuration file(s), should notify the service to restart.

I guess Chef is clever enough not to restart the service it just started. (I didn't pay attention to that before, but it seems to me that I would have, if there were unnecessary restarts)

Answer 4

Chef will only execute an action per resource, regardless of the number of times it's notified. It is, however, specific to each action. So if you send it a action :restart and also a action :start notification, then both will be executed in the order in which Chef encountered them. So what you are seeing with your code example is the initial :start notification from the execute block, followed by the template :restart notifications.

You should be able to avoid this by restructuring your recipe a bit and using subscribes if I'm following correctly.

package 'packagename' do
  ...
end

execute "a command from package which generates and enables the init script" do
  not_if File.exists?('/etc/init.d/myservice_script')
end

template 'configfile1'
  notifies :restart, 'service[myservice]'
end
...
template 'configfileN'
  notifies :restart, 'service[myservice]'
end

service 'myservice' do
  supports :status => true, :start => true, :stop => true, :restart => true
  subscribes :run, 'execute[a command from package which generates and enables the init script]', :immediately
  action :start
end

execute "a command that should run once every time, that requires service to be running"

First, we move the service resource to the bottom of the recipe so that it is only called once all other resources have been evaluated. Then we change the execute resource to subscribe to the :start action of the service[myservice] resource so that it will only ever execute this command when the :start action is called. We also add a bit of idempotency in there to make sure it doesn't just re-create the init script every run.

Finally, we add in all the templates with the normal notifications for service[myservice].