Mostly read only use of DETS

Question

So I have been using ETS - works great. However, I use it as a cache of route data - which I load when the module loads, and save when a change is made (it is read far more than written).

I was thinking that DETS would make things much cleaner - I don't have to worry about managing the persistence of the table. Would this be a good use of DETS? (size isn't an issue, mostly concerned about major increase in read performance - all the data can easily fit in memory).

Answer 1

DETS is much much slower than ETS in most cases but I guess that if your data size is small then most if it will be cached by the disk cache and will be faster to retrieve the second time you read it, so the best thing is the try it with your type of use

But have you considered using Mnesia instead of ETS directly? With the Mnesia table type disc_copies you will get the speed of ETS and still persistency

Answer 2

Straight from the Dets man page:

"Since all operations performed by Dets are disk operations, it is important to realize that a single look-up operation involves a series of disk seek and read operations. For this reason, the Dets functions are much slower than the corresponding Ets functions, although Dets exports a similar interface."

That is, for a read-mostly storage, Dets is not an optimal choice.

(I must admit I find this design decision weird - A better implementation should cache recent lookups. However, since that Ets and Dets are basic facilities in Erlang, I guess that the implementors left that optimization for the users.)

Answer 3

For write rarely, read many data that you're okay keeping in memory, check out a 'Mochiglobal'. Mochiweb has a neat module that abuses Erlang's shared heap for module constants via the code management system to provide super-fast access to terms by generating modules with given values as constants on the fly.

Riak uses a Mochiglobal for the ring state, IIRC.

Hers's the source:

https://github.com/mochi/mochiweb/blob/master/src/mochiglobal.erl

There's absolutely no persistence here, of course, but if you're looking to heavily optimize reads, you can't get much better than this.