Erlang vs The Real/Outside world, how to communicate?

Question

I'm learning erlang and I'm very fascinated by the mnesia db. I want to build some real world application in C# / F# using erlang as backend.

I'm searching for a good solution to communicate with erlang nodes from the outside world.

What I found so far:

(A) OTP.net, an opensource .net library implementing the 'native' erlang comunication protocol

Problems here:

• The library is not very mature
• I don't like the object model ported from Java (too many almost exact replicas of BCL classes)
• I don't like the threading model use for connections.
• Many open TCP ports are required
• Lack of security

(B) Use ports / sockets in erlang and implement a custom protocol.

Problems here:

• I don't have any experience
• Hard to maintain / expand for future versions

Do you have any advice, experience in this topic?

Should I work on the OTP.net library to make it fit my needs or try to implement a new protocol from scratch?

What about a JSON or REST solution? Is there any erlang library that would do the trick?

Answer 1

The port/socket solution is a good idea and is not hard as it may seem. Google's protocol buffers is just what you need. It is very easy to use, efficient and maintainable. It has implementations for C#, erlang, java, python and many more (See OtherLanguages and developer guide)

You can use protocol buffers to define the request and response protocol structure. Then use it to communicate between erlang and any other supported language. The tutorial will explain it all. After that all you need to do is send the response over the port.

The advantage of this approach is that:

1. You can easily extend and change the protocol in the future
2. It is much more efficient than the REST approach
3. It is currently used by Google for almost all of its internal RPC protocols and file formats

Answer 2

If you want to implement a REST API in Erlang there is only one thing to do. Use the excellent MochiWeb Kit to build your own HTTP server that implements your protocol.

Don't panic, it really is easier than it would appear.

There are a number of tutorials about how to do it including a screencast set from the Pragmatic Programmers.

It comes with a complete set of json libraries, so you'll be fine!

Answer 3

Sure, you can do REST with Erlang, see e.g. http://www.infoq.com/articles/vinoski-erlang-rest - if appropriate for your apps' needs, REST is an excellent approach. (Pycon Italia Tre, next week in Florence, has sessions on Erlang/Python cooperation, see www.pycon.it if you're near Tuscany;-).

Answer 4

There is also a JSON library for Erlang, which you might want to look into. I haven't used it, so I can't say anything about it from experience.

Answer 5

While I agree that some REST solution is useful, whether you use Yaws or Mochikit, you will find yourself trying to define some intermediate "language" in order to generate queries that Mnesia would be able to process.

Therefore I offer this advice; whatever project you have in mind for yourself, just implement it in erlang and use the tools available. You'll be rewarded in many ways.

Then again you can always try CouchDB.

Answer 6

We do this using YAWS and a simple http request/response implementation at the client side. The YAWS implementation simply delegates the call to the appropriate gen_server process after extracting the arguments.

The only downside with this approach is that it's not that quick (the Google protocol buffers would be better) - and by keeping the connection "alive" in HTTP parlance helped reduce all of the setup cost and reduces the number of stale socket connections. If you're returning large sets of data you have to get a bit more creative with streaming the results back. For most of our data requests that was not an issue - the response could easily fit into memory.

Some upsides if raw performance of the protocol is not that much of an issue:

• You can hook into a security model (HTTPS or authentication)
• You can call the API from anything that can make a web request (we had lots of old perl code and Excel sheets hanging around - sorting them out was trivial)

Answer 7

We have since re-written OTP.NET

This library is many times more mature, as it has been re-written from scratch for .NET/CLR, (unlike its predecessor that was just converted from Java)

Take a look at this post:

http://blog.aumcode.com/2013/10/nfx-native-interoperability-of-net-with.html