Event Driven Microservice Architecture & Dealing with synchronous external neighboring systems

Question

Let’s say I have an event driven microservice system - a microservice cluster - that takes orders and works with them. Orders are placed by an external neighbor system that is working synchronously via a REST endpoint provided by one of my microservices. This service is acting as a facade to keep all the synchronous stuff on the outside of my cluster since every communication on the inside is message based (asynchronous). Incoming orders will be stored in a database by another of my microservices who reacts to an event message send by the facade microservice.

Now, suppose there is a requirement to deal with incoming cancellation for orders and these cancellations are also coming in via REST. For valid cancellations an order has been placed before and therefore is stored in the database of my cluster.

If I wouldn’t try to design event driven I would think of something like: If a cancellation is coming in via REST, I synchronously check if a corresponding order for that cancellation is in my database. If so my synchronous response on the REST call would be either a 200 if everything is OK and if not, I would reject the invalid cancellation.

But since I am trying to design an event driven system I don’t know how to deal with this. I think, maybe I would have another facade microservice that would provide the REST endpoint for the cancellation. And on reception of a cancellation it should also send a message into a specific queue/topic to start working asynchronously from there. But how do I deal with the fact that I need an information from the database to check if a cancellation is invalid and this new facade service has no access to my clusters database? Do I have to always accept cancellations and tell the outside world that everything is OK, even if I don’t know if that is true? And if the other system needs to know about invalid cancellations, do I have to give them access so that they could listen to one of my topics/queues, so that they then can react to an event message that travels through my cluster containing the information if a cancellation was valid? Or do I have to set up another service where the outside world can ask if a cancellation was valid? And what if I can’t talk to the maintainers of the foreign system or they don’t want to change their systems api? Or do I have to give my database accessing microservice an internally accessible REST endpoint that is accessed by my facade microservice synchronously?

How do I design this while keeping my microservices scalable, resilient, loosely-coupled, etc.?

I hope I asked my question in a comprehensible way. Maybe am I thinking completely in the wrong way and/or missing something about event driven architecture?

Hopefully someone here can give me an solution, an idea or a hint into the right direction.

Many thanks in advance!

Answer 1

Synchronous calls are only a problem when the work required is slow.

Since you just have to check a db, You can have a queue for cancellation validation requests with an RPC style return queue and call it from your REST service, while keeping the incoming request waiting.

Where you might end up with issues is when the Order Placement message is still on the queue to be processed and you get a cancel for it. It doesn't seem unsolvable though.

If the cancel request processing was slow, well then yes, you would have no choice but to adapt your workflow to an async one.

My general advice would be: If you are using message queues, make your microservices a bit bigger than you would otherwise. Just to keep the number of internal message types down to a manageable level.

I probably would have the rest service check the db directly. If you need a message to trigger other stuff you can always fire off a CancellationRequestProccessed or something at the same time as doing the actual work.