Event-driven architecture and synchronization with a “First-timer microservice”

Question

I'm working on a distributed application and getting deeper into Event-Driven architectures with Microservices. Let's say I have been running two Microservices, CustomerService and AccountService. They both have their own database, and they've been keeping in sync with an event-style messaging with RabbitMQ in between.

For example, when a new user is created in the CustomerService, a CustomerCreated event is sent to RabbitMQ. The AccountService has a subscriber that listens to this event and immediately creates an account for this user.

Now, several years in the future, I want have a new service, MusicListService, that whenever a new CustomerCreated event is raised, bind a Music playlist to that user. Obviously the previously created users need to have a playlist binded to them as well. But since this service is brand-new, it didn't receive the past CustomerCreated from the former years, so no playlist will be binded for the former users unless I create some kind of out-of-band script that brings the database to a proper, up-to-today state.

What would be the best start-up strategy to a new Microservice so that it keeps in sync with the ones that were already existing?

Answer 1

Generally you take a snapshot in time of existing data and either process that snapshot via an external process or by replaying the snapshot’s messages over the bus/queue.

Practically, you’ll need to account for out of order messages, as well as duplicate messages because the snapshot and normal message stream need to overlap a little to ensure there’s no gaps (and thus missing data). But you should already be accommodating for that since you’re using a messaging system.