Official/Advised/Allowable maximum number of records (or RAM Table Size) in ETS / Mnesia RAM Table

Question

Erlang has been a good application to many "New Age" (obscenely fast and fault tolerant ) Messaging Systems. Examples can be RabbitMQ, ejabberd, WhatsApp for Mobile e.t.c. Regardless of how a messaging system is built, it will need to know what sessions are going on, who is logged on, e.t.c. I assume that holding data IN-MEMORY is unavoidable for such systems.

For Erlang, what is the possible maximum, OR what is the allowed maximum OR what is the officially advised maximum number of records in an ETS table or mnesia RAM table ? Is there a certain Table size that is known to crash the whole VM if exceeded ? Do ETS Tables or Mnesia RAM Tables grow to any amount as long as the underlying Operating System allows it ? I have heard of Mnesia Table Limit of 2 or 4GB, but i thought that this is for Disk tables.

A Typical example of future applications that will need to process large amounts of data IN-MEMORY are OLAP Systems, Real Time analytics, Social Gaming e.t.c. Thinking in this line has motivated this question.

Answer 1

mnesia is a wrapping layer of ETS and DETS tables. For a 32bit system a maximum value of a ETS table is about 3.5GB. For a 64bit system maximum value is much more, so technically it is limited by RAM available for OS. For DETS (and it is very stupid) limit is 2GB.

I don't have any benchmarks with other key-value storage systems, but remember, that ETS stores Erlang terms directly without any encoding/decoding.