Microservices - should there be multiple instances of the same service?

Question

I've been doing some research on microservices, and one question I've had that is not well addressed is whether there should be multiple instances of a microservice, and if so, how to deal with this?

Suppose for example I have a system of two microservices, A and B where A depends on B. Looking at B, is B supposed to be a single instance always, and handle everything A throws at it, or is it possible to create a pool of service Bs, so they can handle the load?

If the latter, how does one manage a pool of microservice B instances, with respect to load balancing and their data ownership?

Answer 1

Answer mostly depends on your scalability/uptime/failover requirements. If it's necessary to have highly resilient architecture, then yes, you need to have multiple instances (sometimes even distributed across multiple data centers).

This is actually one of the benefits of microservices - you can scale different services independently - some of them are more performance critical than others so you create more instances of these. Contrast this with monolithic applications where you can scale just the whole monolith (ignoring the fact different requirements for different parts of the monolith).

If you're lucky, then you microservice is stateless - all instances are completely isolated. This is great because you can scale them almost infinitely. But often you have some state which needs to be distributed among the individual nodes. If you have relational database, you can have all nodes connect to the same DB instance, but this usually doesn't scale well. You can then setup read replicas (slaves), or even multi-master replication (read and writes possible from several DB instances which then sync), but you can still hit some limits (depending on the application and scalability requirements). Recently there's hype for distributed key-value stores - e.g. Hazelcast which can have a lot of nodes (even one per application instance) which then synchronize. This usually scales better than relation DBs, but data is usually not persistent and lost after system crash.

When you have multiple application instances, it's a good idea to hide this fact from the client - you should have only one endpoint which is served by a load balancer (e.g. HAProxy) which then directs the traffic to the actual application instances. These instances can be added/removed based on the load dynamically, the endpoint never changes.

Answer 2

Why not. If the microservices are stateless or have a shared database backed state then you should be able to have multiple instances of the services behind a network load balancer (F5 big-ip or Microsoft NLB as an example). I am assuming the microservices have a http api (REST). The answer is slightly more complicated if the microservices are integrated over a servicebus but even then you can have multiple instances.

Answer 3

Services in general should be stateless, if possible. This allows you to harness one of their most powerful advantages: horizontal scalability. If you're on AWS, for example, you can set up auto-scale groups that simply create more servers when necessary and destroy them afterwards, with no loss of information or requests.