Service-Oriented Architecture with Erlang/OTP: how to use messaging properly

Question

The answers to my previous question about the database access and service abstraction helped me to understand that in SOA a database service should provide a public API, but not a direct access (SQL or via ORM) to the database.

Now, if I want to create a (SOA) service in an Erlang/OTP application, I think it should be a gen_server or other OTP behaviour, because in that case the API will be easily defined and the service itself can be easily incorporated into the application.

But how should I pass data from the service to its consumer?

I know that in Ruby on Rails I can do something like

persons = Person.find(all)

Then I can just return persons to the service consumer.

In Erlang/OTP if persons is a few thousands records, should I pass it in the gen_server return statement? This means that the gen_server will send a message to its caller (service consumer) and the message will be essentially copied in the Erlang VM. I tend to think that it is not a proper use of Erlang message passing.

So my question is:

• How can I create a database service (SOA) in Erlang/OTP?
• Should the API of such a service use the OTP standard mechanisms (like gen_server return message) or anything else?

Answer 1

Not all Erlang Applications need to have messaging ! Also, Not all Erlang applications need to use gen_servers. For applications that involve a connection e.g. a web application, lets analyse, really, where u would need a gen_server.

A web server (Yaws) will essentially spawn a process to handle each incoming request. Now, in your application, the following things may occur:

1. You may also be spawning another process to work on the request and then send 
the answer as a message to the first process
2. You could just be processing the request through a bunch of sequential code 
in different modules, say direct to mnesia or through an odbc connection e.t.c.

Now, in case of 1 , you are already increasing the number of processes per request. Most applications will be parallel enough with case 2. How does a gen_server come in on all this ?

A gen_server is an abstraction of many things. However, still, a gen_server handles requests one-by-one , creating a point of failure as well. Ofcourse, when it fails, there is a restart strategy which will bring it up. So If you have a gen_server standing in between yaws and say mnesia or an odbc connection to MySQL / Oracle / e.t.c. then, the parallelisation done by yaws is a waste of time because, the gen_server is serving those processes sequentially. Now, however, if each process was going through a bunch of sequential code to get its answer, the benefits are enormous.

Actually, i have created some SOA applications before, but they are RESTFUL in nature. I found that actually, using say, yaws appmods (they are essentially RESTFUL already), would be wasteful, if a have a gen_server in their path to information in a database. Removing the gen_server improved performance in terms of speed and stanility of the application. Some times , the mailbox / message box of your gen_server can grow too big and it will slow down the entire system because processes will be waiting for answers. Another thing, because messages are copied all over the place in an erlang VM, memory becomes a problem. As a reference example of exactly your problem, open this book (Franceso and Simpsons Erlang Book) and look at page 110 , under the title A Case Study on Concurrency-Oriented Programming, they show you process design patterns, the bad way and the good way.