dev opsworks - updating the app, will it drainstop apache

Question

I have an autoscaled environment at my production, which is currently a havoc when we update build on it, so we thought we better move to dev opsworks at AWS to make the process more easy for us. We can't afford a downtime, not now not ever, never ever; a second worth of loss while updating a build and may be restarting apache costs a fortune.

We can't possibly afford to just let our machine be terminated by autoscale policy when a new update comes in with new AMI based ec2 machine, actually when autoscale terminates a machine under any circumstances it doesn't care for your running requests on that machine, it just shuts it down while what it should rather do is a graceful shutdown, by something like drainstop on apache, so it could first at least finish the work in hand.

now that opsworks is here, and we are planning to use it to update our builds more automagically, will the new update push run the recipes again, in fact this paragraph which i just read worries me more, does it mean that it won't update the build automatically on new instances.

After you have modified the app settings, you must deploy the app. When you first deploy an app, the Deploy recipes download the code and related files to the app server instances, which then run the local copy. If you modify the app in the repository, you must ensure that the updated code and related files are installed on your app server instances. AWS OpsWorks automatically deploys the current app version to new instances when they are started. For existing instances, however, the situation is different:

You must manually deploy the updated app to online instances.

You do not have to deploy the updated app to offline instance store-backed instances, including load-based and time-based instances; AWS OpsWorks automatically deploys the latest app version when they are restarted.

You must restart offline EBS-backed 24/7 instances and manually deploy the app; AWS OpsWorks does not run the Deploy recipes on these instances when they are restarted.

You cannot restart offline EBS-backed load-based and time-based instances, so the simplest approach is to delete the offline instances and add new instances to replace them.

Because they are now new instances, AWS OpsWorks will automatically deploy the current app version when they are started.

Answer 1

First of all, let me state that I've started looking into OpsWorks just about 2 weeks ago. So I'm far from being a pro. But here's my understanding how it works:

We need to differentiate between instances that are instance store backed, and instances that are EBS backed:

• The instance store disappears together with the instance once it's shut down. Therefore, bringing it up again, starts from zero. It has to download the latest app again and will deploy that.
• For EBS backed instances the deployed code remains intact (persisted) exceeding the lifetime of the instance to which it is attached. Therefore, bringing an EBS backed instance back to life, will not update your app automatically. The old version remains deployed.

So your first decision needs to be what instance type to use. It is generally a good idea to have the same version of your app on all instances. Therefore I would suggest going with EBS-backed instances which will not automatically deploy new versions when booting up. In this case, deploying a new version would mean to bring up brand new instances that will be running the new code automatically (as they are new), and then destroying the old instances. You will have a very short time during which old and new code will run side by side.

However, if you prefer to have always the very latest version deployed and can afford risking discrepancies between the individual instances for an extended period of time (e.g. having different app versions deployed depending on when an instance was originally started), then instance store backed might be your choice. Every time a new instance spins up, the latest and greatest code will be deployed. If you want to update existing ones, just bring up new instances instead and kill the existing ones.

Both strategies should give you the desired effect of zero downtime. The difference is on when and how the latest code is being deployed. Combine this with HAProxy to have better control which servers will be used. You can gradually move traffic from old instances to new instances for example.