Dynamic cluster resizing on the cloud

Question

I'm a Java developer and am considering AWS, Rackspace and Heroku as potential cloud providers for my app. Having only worked with traditional web hosts before (virtual dedicated server plans), I'm sort of overwhelmed.

My understanding is that, with the cloud, you can spawn/kill virtual appliances (virtual machines with preconfigured software stacks, such as your app servers, database servers, etc.) based on demand. So if its midnight or a holiday and no one is using my app, then I can cut back on the number of running instances and save some money. But, if its a Saturday morning and I'm getting dangerously close to peak load, I can spawn new servers and rest assured that my app isn't going to crash.

So to begin with if my understanding is incorrect or flawed in any way, please begin by correcting me!

Assuming I'm more or less correct about clouds, then it makes sense that I should build my own monitor that will watch for sudden increases/decreases/patterns in app usage (I assume based on network traffic and bandwidth?!). When traffic grows to a certain point, I should instantiate more VMs, and vice versa when traffic slows down.

Since JClouds allows you to do this my monitor would just use the JClouds API to spawn/kill servers.

What I'm choking on is: what algorithm/formula should I use to tell my monitor when to spawn/kill VMs? I wrote a tentative algorithm that basically ensures that I have double the "horsepower" (VMs) that I need at all times. The monitor polls each node about every 10 seconds and, if the current load on the system has changed by a certain delta, it allows the ClusterResizer to kick in and determine if we need more/less nodes.

Although I've spent a few hours designing the outline/skeleton of this dynamic resizing system, I wanted to stop before I got too far ahead of myself and do a sanity check here in case I'm doing something unnecessary, wrong or just plain crazy!

How do other cloud apps monitor and resize their VMs? What algorithms or best practices should I be familiar with? Thanks in advance!

Answer 1

On Amazon EC2 you don't need to implement the monitoring yourself, there is a feature called cloudwatch that does elastic scaling for you - scales up automatically when load increases and scales back down when it decreases. So you won't need to implement the monitoring service yourself. On RackSpace there is a third-party solution for auto-scaling called Scalr. There may also be a native RackSpace feature equivalent to Amazon's CloudWatch, you should investigate to save yourself building this infrastructure from scratch.