What is the best way for distributed processes to communicate asynchronously?

Question

I'm developing an application in which distributed components talk to one another over a network, in an asynchronous, pub/sub kind of way.

For this, I like the idea of sending XML over sockets - it's asynchronous, I don't need a server of any kind, and it can work locally or over a network. I would have to roll my own pub/sub mechanism, which probably won't be too onerous - I could use the IP address as the id of the subscriber. The implementation language will be C++. I've used this method before with good results. But, I wonder if there is a better way? Any advice welcome.

Edit: I haven't made a decision yet, but I'm interested in cparcode's comment on protobuf:

"I never understood the popularity of using XML for network comms. Any way, I realize that the protocol is only part of your problem but on that subject, there's Google's protobuf too: http://code.google.com/p/protobuf/ – caparcode (Mar 11 at 1:01)"

Answer 1

Instead of using XML it might be best to use an existing message passing framework. Have a look at libt2n or d-bus

But it you have used your own socket implementation in the past with good results i don't see a reason why you should change. If it ain't broke don't fix it :)

Answer 2

Instead of just straight sockets, you might consider a solution like AMQP.

When you're talking about using socket connections for a pub/sub interface, that usually means point-to-point communication, which isn't always a scalable solution. AMQP really addresses this sort of pub/sub problem. It's free, it's open-source, and it works.

Given that they've already solved the pub/sub problem, you might want to leverage off their work instead of doing it yourself.

I recommend OpenAMQ.

That being said, it really depends on what sort of environment you're working in. AMQP requires an AMQP broker (an application responsible for routing messages) to be running somewhere in the system.

Answer 3

Take a look at ActiveMQ. It's a JMS provider and also has bindings in lots of other languages, works on several transports, and has a selection of protocols that it uses for its messages.

Answer 4

RabbitMQ is also an AMQP broker, that supports other protocols like XMPP, HTTP, STOMP, SMTP as well, using adaptors.

Answer 5

Especially if you have done it before, and it still meets your needs, stick with it.

Answer 6

How about embedding a web server in your app? EasyHTTPD is open source, C++, and pretty simple to use. The asynchronous part is easy enough. You either spin up a new thread on the sender or the receiver.

Answer 7

If you're looking for XML, and cross-platform to boot, you may consider an XML-RPC mechanism like SOAP (I've used gSoap for communication between a C server on linux with a C++ client on win32, and it worked fine once you figure it all out).

Answer 8

If you are not heavily inclined towards XML and you want something simple and efficient then consider looking at YAMI library. It is rather simple to use and the given link contains enough documentation, rationale and examples to get started.