AWS SQS + autoscale

Question

Assuming that I have a queue and multiple instances in autoscaling group.

For scaling up case, it's quite easy to determine.If the length of the queue grows, autoscaling group will spawn new instances.

For scaling down case, it's a bit tricky here. If the length of the queue shrinks, autoscaling group will terminate the instances. It sounds obvious, but the question is: what happens if the instances which are still processing messages being terminated?

Of course we can use some metrics like CPU Utilisation, Disk Read/Write, etc to check. But I don't think it's a good idea. I'm thinking about a central place where instances will be registered for whether they are processing or not, so that the free ones can be determined and so terminated properly.

Any thoughts for this? Thanks.

Answer 1

The accepted answer on this thread:

Amazon Auto Scaling API for Job Servers

gives you two possibilities to handle your situation. One of them should work for you. Also keep in mind, that you don't necessarily want to kill and instance as soon as there is no work - when they spin up, you are going to pay for the whole hour wether you use 59 minutes or 1 minute, so you may want to build that into your solution - spin up instances fast, turn them off slowly.