Azure SignalR and backplanes for inter role communication

Question

I am currently using signalR on Azure Websites with a single instance to push data to clients. No problems.

We're splitting our project into separate web/worker and wcf roles so we can scale them independently.

The site will work like this.

Scenario A

User submits some data to web role and it gets put in a service bus queue ready for worker A, sends a message to worker A that a new item has been added in case it's idle (to save polling). When worker A has processed it, sends a message back to web roles which pushes out to particular clients.

scenario B

receive data in wcf role and it gets put in a different service bus queue ready for worker B, wcf role sends message to worker B that a new item has been added in case it's idle. When worker B has processed it, sends a message to web roles and pushes it out to particular clients.

illustrated badly below:

I am going to enable signalR service bus backplane for the web roles to users. What i'm not sure about is how to get my roles communicating between each other.

I'll need:

web role => worker A

worker A => web role

wcf role => worker B

worker B => web role

Am I creating hubs on web, worker A and worker B all with service bus topics? And then connecting somehow with the signalr .net clients? How do I make sure it goes to all instances of the web role without exposing it publicly?

For some reason it seems simple for hundreds of clients to connect via JavaScript to my web role hub but try and connect some internal ones and I can't quite figure it out.

Answer 1

If anyones interested... What I ended up doing is this:

I created hubs on both the Web and Wcf role. The web role has a connection that allows javascript proxies at /signalr and the web and wcf role had one that didn't at /signalr-internal.

I used the Azure Service Bus as a backplane and let it handle both the web and wcf hubs automatically with no extra tinkering.

In the signalR authentication I probed to see where the connection was coming from (i.e an internal endpoint or the external ssl endpoing and denied / allowed access to particular hubs based on this. This allowed me to use the .net signalr clients on my workers that automatically connect / reconnect etc.

This ended up working nicely with no issues as of yet and it was simple to implement. I'll update if I run into any problems.

EDIT #1:

DO NOT USE THIS METHOD! Everything works splendidly until you actually deploy it into a live environment and then you get a host of issues that made me want to tear my hair out.

What I actually ended up doing (which work perfectly in live) was to use service bus Topics and create subscriptions to them for the listeners. This creates TCP connections and allows your communication to stay 100% internally without any crazy transport or boundary problems.

EDIT #2:

Since this post, Event Hubs were release and we switched over and never looked back. see last comment

Answer 2

Peter, realistically to get this approach to work you would need to switch to Web Roles or IIS hosted on an IaaS VM.

Currently Websites don't support Azure Virtual Networks which is the only way to enable private network inter-connectivity between instances on Azure.

You can add VMs, Web and Worker Roles to a Virual Network which should provide you with the access you're looking for without needing to expose everything via public endpoints.