Using Python for Erlang-like things: what goes where?

Question

I do not yet know much about it, but I've heard that Erlang is somehow special in processing network applications; as I understand it, this comes from some early design decisions that allow it to have thousands of concurrent processes with limited overhead.

I'm essentially a python programmer, and I've enjoyed using 0mq lately for distributed computing like this, until I came across this article criticizing this article.

What is it that Erlang does that is so special for networks/concurrency, and which python library, or set of libraries, attempt to replicate/replace which parts of it? I'm having trouble understanding /how/ each of these items fit in to the distributed, networked programming picture:

• 0mq wrappers to python
• stackless python
• Twisted Python
• Tornado

(I understand that Celery and RabbitMQ are slightly different beasts in that they are message passing, but they mandate a queue system, which is a particular type of distributed computing.) How do these "map on" to which functionality, and when should they be used?

I have also reviewed the following:

• https://stackoverflow.com/questions/5519513/erlang-like-concurrency-for-python which is 2 years old.
• https://stackoverflow.com/questions/3423447/which-actor-model-library-framework-for-python-and-erlang-like which assumes the actor model is secret sauce of Erlang---I do not know enough to know that this assumption is the right question.

Answer 1

Your question is very broad, I'll try to answer few points briefly. Concurrency in any language is not perfect, every language, model, framework, library comes with limitation, they can be perfect for a subset of situations but in general nothing is perfect.

When we compare two languages for concurrency we have to look all aspects. There are number of reasons why Erlang is popular and different then python.

Strong concurrency can help make good, scale-able, robust and fault-tolerant network apps and Erlang have these qualities. I have very little knowledge of python but as far as I know the concurrency in python is not as good as Erlang, see point, as under, to know why I am saying that.

Let me present you few points about Erlang and concurrency in general.

1. Erlang use OTP library which is very popular and useful library for network related operations. This is the main reason erlang is considered better for network apps.
2. Green threads are better choice than native threads if implemented carefully and correctly. To the best of my knowledge Erlang's green processors (similar but better than green threads) are lightweight and best. Erlang provide its own implementation for process scheduling and do not use operating system for this purpose while most of other languages rely on OS.
3. Erlang itroduced the concept of actors, which is another abstraction on top of green processes. Erlang can create millions of actors with very small memory foot-print, theses actors communicate Asynchronously, which means they donot need locks and hence perform better in network applications.
4. Erlang is roust and fault-tolerant. It provided the concept of supervisors which keep an eye on actors. If any actor fails to perform its jobs it is its supervisor deal with actors. Supervisors can either terminate that actor and restart it or supervisors can terminate all of its actors and restart.
5. Erlang is a functional languages and inherently functional languages are more suitable for concurrency then imperative (or any other paradigm) languages. One of the many reasons is immutability, which eliminate fear of race conditions, deadlocks etc.

There are many other features which makes Erlang very suitable for concurrency and hence suitable for network apps. Many other concurrent languages picked up concepts from Erlang which were basically introduced by Erlang.

actor model is secret sauce of Erlang

That's not true. Scala (via by Akka) and many other languages (or libraries) support actors (similar to Erlang)