How many Topics (queues) to practically use in an ESB

Question

In all the Service Bus examples and tutorials I've seen, they cover publishing a single message to a single Topic that is picked up by a single listening Subscriber. Whilst this is good for a simple practical overview, it doesn't really explain how Topics should be used in the real world.

I have a large ecommerce application that spans many subdomains, such as Order, Customer and Product management to name a few, and will be using an ESB to communicate events between these disparate systems. I will be using Pub-Sub via Topics.

Initially I was just using a single Topic per each different message type I was sending, however I could see that this was just going to get unmanageable as the system would just have too many types of messages to send to justify a 1-to-1 Message Type to Topic. e.g. ProductCreate Topic, ProductDelete Topic.

Then I thought that I could use a Topic per aggregate in the system. However because of the multiple bounded contexts in the domain, an aggregate in one system could be different to an aggregate, with the same name, in another system. So this implementation might be quite confusing.

So having a Topic per ubiquitous model seems like a better solution, and this can be based off the Canonical Data Model that is used to transform messages across the service bus. Is this a common approach ? Or is dividing the Topics up per functional concern a better method ?

I can see that there are going to be compromises to whichever implementation I pick, so I'm looking for good tips, historic examples of success and pitfalls to avoid.

Answer 1

We struggled with the very same choice / challenge. Mapping only 1 message type to ASB's topic is just not practical. But making it too generic (e.g. sending strings in message's body, using custom and dynamic serialization everywhere) was not appealing either.

We ended up building a light weight framework on top of ASB. It allows sending and receiving multiple types of messages over the same topic, and all of its subscriptions, while using ASB's native serialization. The trick was to capture the type of the class that is being sent in message's ContentType. Multiple subscription clients (each aware of what type they care about) would be listening on the same topic at the same time. A specific client would only handle messages of its type, and ignore (Abandon) others. Our framework uses reflection behind the scenes to call message handlers - no way around that. But it's abstracted away from the code which relies on it - everything is strongly typed from client's POV. This works both for partitioned and non partitioned topics.

I do not have data yet on how much overhead this approach can introduce, and what it means for high volume messaging systems. We are using ASB from on premise application, where ASB latency is high to begin with. Should know more in a few months after we move to Azure completely.

BTW not 100% sure what you meant by "aggregate" or "canonical data model", but hope this helps.