What is the scope of blue/green deployments in large-scale environments?

Question

In my past experience, the way we enabled blue/green style deployments was to provision some servers for the new release that were copies of that app's production environment servers, deploy the new app version to them, test, then switch over the traffic routing to point to the new servers. When I suggested this in my new team, they said their impression was that blue/green should be across the entire IT environment - you have a copy of all the apps and servers duplicated at all times, and switch back and forth as changes are made.

This surprised me a bit as it seemed like a lot of overhead to switch the routing back and forth for hundreds of apps on hundreds of servers every time a single app is updated. I went back and re-read the Martin Fowler article and it's unclear to me what the usual scope is; he repeatedly talks about identical environments, but the sample picture seems to just be a single app stack (one blue web server, app server, database, and one each for green). I found similar descriptions elsewhere around the web; everyone talks about a duplicate environment but then gives an example focused on just a single app.

So, how have you setup blue/green deployments? Am I missing something by having focused on the individual app/service level?

Answer 1

I'm unsure why you would want to have a blue/green deployment that is triggered for all servers (assuming each application is running on a separate server) when only a single application was updated. I'm hard pressed to see what the benefits of such a system would be and can only imagine that it would give more opportunity for things to go wrong while switching over on servers where no changes were made.

There are 3 reasons to use a blue/green deployment as I see it:

1. You can test the update on the new server before it is made live.
2. No down time when switching to a new version.
3. You can easily and quickly roll back to the previous version if something doesn't work with the new version.

There would be no benefit in switching an application that wasn't updated and it would remove the 3rd benefit since both the blue and green servers would now have the latest version.

I think you somewhat answered you own question in the last sentence. Each individual app/service should be viewed on its own and have its own blue green deployment.

Answer 2

Martin Fowler's article was written before Docker existed. Admittedly, my own experience biases me towards containers, but in the last couple of years I believe it has become more and more rare to do things like blue/green deployments without containers, especially in large scale environments. I haven't seen any tutorials recently about doing blue/green at the server level, like you describe, let alone the way your colleagues describe.

Using containers lets you use a scheduler like Nomad or Kubernetes to bring up the next version of your app on the same servers, while using the same web server and db containers if they haven't changed, perform health checks and/or other tests on them, then roll back or promote.