How do I not lose the “reactive” nature of my services, simply because one of the services isn't reactive?

Question

I've lately been very interested in reactive microservice design with streaming and event-driven architectures.

When one writes (i.e., manages) services, this paradigm works extremely well by simply responding to published events on topics.

However, we don't always control the implementation of all the services with which we interact. For example, consider a commercial service which exposes a black-box endpoint which does some asynchronous work and returns an ID for that job. It also exposes an endpoint for retrieving the status of that job by ID. For sake of simplicity, we'll assume these two endpoints and a "get results" endpoint are the only exposed endpoints.

In this case, we are left in a situation where we must necessarily poll the service to check the status of the work. Are there established patterns for doing this? I'd guess a supervisor which polls on a timer and publishes the job status might be a "fine" approach, but I'm wondering if there are other battle-tested approaches.

Answer 1

Guess it's a bit late but here is what I would do : I would put a wrapper around your part and the blackbox.

This wrapper will handle the creation of the jobs and the pulling. You have to make it qo only him can pull on that service. The wrapper will publish event on each finish jobs.

To avoid DOS, you should have a limit of how many jobs you can ask to be processed in the same time. If you don't, come up with a reasonable numberand have your wrapper "queing" or refusing job if the number of jobs running in the black box is already at the defined limit. And by "queing", if you choose that, I mean that your wrapper will have it's on queue of job to process when the black box is already full. That queue will get processed every time there is free space on the black box.

The gist of it is basically that if you can't make some subsystem that is compatible with your system (legacy, foreign service,...), you wrap in something that make it so, ans nothing else than the wrapper can use the underling subsystem.