Microservices and isolated data storage models [duplicate]

Question

This question already has answers here:

Why is it so bad to read data from a database “owned” by a different microservice (5 answers)

Closed 4 years ago.

I have been looking at Microservices for a while now. The concept is not new but it's communicated in a lightweight manner. So, I am very excited about this.

However, there is a question that I am not sure what the answer is: is each microservice supposed to have its own isolated data storage model completely? Consider the below:

Note: This is probably not a good example as the seperation here is done based on read/writes whereas the good approach would be to separate the business concerns.

• products-write-service: responsible for dealing with creation and edits of the products. Knows MongoDB and writes data to there.
• products-lookup-service: responsible for dealing with retrieving products by their identifiers, providing basic listing of products based on categories, etc. Knows MongoDB and reads data from there.
• products-search-service: responsible for dealing with search requests on products. Knows Elasticsearch and reads data from there.

Here, we have two separate data storage techs between three services. However:

• products-write-service and products-lookup-service works on the same data storage model. There is a tried relationship between two services.
• at the end of the day, there should be a process to do ETL from MongoDB to Elasticsearch for products-search-service to function. In some sense, three microservices have their own data storage model but there is a middle process which knows about these two models.

What do you say about this model? Is the separation completely wrong here?

Answer 1

I think that each service should have its own storage. Otherwise you're missing a point of (micro-)services - you can't quickly evolve the service if you're limited by storage shared with other service.

Why are you splitting read and write logic into separate services? That's a bit of a smell for me.

There's nothing saying that a service couldn't have multiple storages. Maybe you're looking at a single service with multiple storages? Updating a product could result in simultaneous updates to both MongoDB and ES.

Based on a limited domain knowledge I would provision 2 services: Product Catalog with MongoDB and Product Search with ES as backend. How you transfer data between services is a separate question:

1. ETL as you've suggested (would have to understand both storages - quick and dirty)
2. Domain Events - Product Search service could subscribe to events published from Product Catalog service (that would be my preferred choice)
3. Batch process that would use services' API instead of going directly to the storage (probably better than 1. but will require developing richer APIs)

Answer 2

Well, the best analogy is encapsulation and Data-Hiding in OOP.

The only way you can interact with the state of the object is through its public interface. In the same manner, the only way you should be able to interact with the data of a (micro)service must be through its public API. Having said that, (micro)services often events that other services could subscribe to.

As such, read and write services are very much the same (micro)service as they access the same data source.

So what is the difference with the classic SOA? Well, not a lot here. But if you start breaking Product Catalog into its different aspects then services will be smaller. For example a classic SOA customer service stores everything related to customer but in a Microservice world, the address goes to one service while preferences got to one or more services.

But not all microservices have a data store. Some Microservices are composition services which aggregate the data from multiple service (enrich) and represent a more usable facade. Also some Microservices encapsulate an algorithm and do not have any data as such, perhaps some configuration only.

Hope this helps.

Answer 3

For answer: each service should have own storage mechanism.

Underline informations:

Maybe your micro-service (bounded context) boundaries are wrong.Product write and lookup service might be belong to same boundary but searching is not domain concept so searching might be another micro service boundary. Product lookup service is basic rest service in Product bounded context so it doesn't need to another database.

https://www.thoughtworks.com/insights/blog/domain-driven-design-services-architecture