How to make a fault tolerant system which can immediately handle the situation when a server goes down

Question

Before XYZ.com was down I noticed that my request was being routed to IP address 192.33.31.xxx and when it came up I noticed that my request was routed to IP 50.17.196.xxx , is it some sort of server switching? Isn't Dynamic server switching in case of fault is a way to make fault tolerant system?

Answer 1

Most of the heavy used websites works with load balancers directing the request to multiple servers. There are very simple to very complex logic based on which the routing is defined. For e.g. the logic can be

1. Once the request is handled by one particular server the request from the same IP is handled by the same server OR
2. Use round robing way of handling request ( In this case IP has role to play)
3. route request to the least used server etc.

Website can also use hybrid approach of all the above.

Now if the website you were accessing was using the IP based routing then the case would have been

1. This is your first request. Then you would be assigned a server which will handle your request. (This can be round robin or other logic (for e.g. the server which has less load)
2. Now the load balancer might have logic that next X (say 100) requests or for Y (say 1 hour) time period, all requests from the same IP will be routed to the same server or both
3. So in your case if you make a request to XYZ it routes to some a.b.c.d server and then if you make another call after 1 hour or you make 101'th call you might get a different server which handles your request.

Now if your server went down, then you can have a fallback mechanisms like,

1. If a server do not respond in some pre configured time period, remove that server from live server List in load balancer configuration and reroute the request.
2. Or we can have an idle sitting pre running servers (it is costly but to be followed when each request has to served without fail). As soon as a server goes down, invoke this new server and assign it the ip which was assigned to the old server which went down.

There can be better and much more complex solution for this problem though.