Architecture for communication between multiple backend systems and IoT devices via WebSockets

Question

I have a scenario in which thousands of IoT devices -- each fitted with a sim card -- are geographically distributed and shall be controlled individually.

Therefore each IoT device has a WebSocket client with which it can connect to a central system. Since there might be a lot of communication, the IoT devices connect to a backend WebSocket server component (Gateway) that is horizontally scalable and behind a load balancer.

In the backend landscape there are other components (Master) that want to send commands or queries to the IoT devices based on their UID. Therefore a Master must find the correct Gateway which has an open WebSocket connection with that specific IoT device.

My initial idea now is to introduce a message broker (Kafka?!) between all Master and Gateway components. Each Command or Query message together with the targeted UID is thrown onto the message broker and into a request topic/queue/stream.

Each Gateway reads all request messages and if one Gateway has an active WebSocket session to the IoT device with the given UID, the Command or Query is sent to the IoT device.

I would be happy about a review of my described design! Are there some common heuristics or algorithms (distributed system design) that I could rely on?

I don't want an registration service because it would introduce an extra call or each Master would build its own registry and if a Gateway crashes the registry would need an update.

Answer 1

The approach with a message broker is what SignalR in Net uses, it has multiple options for the message broker. I included link below which describes their approach for reference even if using other technologies. This is what we used with Azure Service Bus as the backplane.

https://docs.microsoft.com/en-us/aspnet/signalr/overview/performance/scaleout-in-signalr