How should two servers communicate

Question

For a new project I need to design a multi-component back end in Python. Initially, it will have two basic components -- a business rules server and a front end that will serve the requests coming from the browser.

|--------------------|
     Business API
|--------------------|
         ||
         ||
         ||
|--------------------|
    Front server
|--------------------|

As this system gets more features I might need to add more servers/components connecting to the Business API.

Now, I have, (after researching a lot) settled on MessagePack for serialization/deserialization purposes.

What I can't decide though is what should be the transport (wire protocol), over which the communication should happen. There's some options:

• raw TCP/IP sockets
• zeromq sockets (zerorpc)
• plain HTTP

I think http would be a good choice to start with -- but how should I send the payload over http? by doing a base64 encode? In that case I would have to do it like this:

on one end
[actual message] -> [msgpack encode] -> [base64 encode]

on the other end
[base64 decode] -> [msgpack decode] ->   [actual message]

Is this acceptable in a system where there's a lot of chatter going on between disparate components? Are there any better solutions? Is there any better way to do it over http?

Note: I can't use plain JSON as I need to transfer binary data - so the serialization library is going to be msgpack.

Answer 1

It really depends on the details of your system.

If your components are all in Python (or Node.js) and all communication happens within your network you can go with ZeroRPC.

If you need to cross firewalls or if you use more exotic languages you can go with HTTP, as Hazzit said in a comment HTTP it is perfectly OK to transfer binary data in the body of an HTTP document, or alternatively you can use Multipart Post encoding.