how to create multiple logical tables in single levelDB instance?

Question

I am working on a Distributed key-value system (or data-store), which uses levelDB as its embedded database library in the back-end.

I want one node/machine to host multiple tables (for the purpose of replication and load-balancing). I understand levelDB has no notion of tables, so I cannot logically partition my data in form of tables (hence cannot use these tables as my basic unit of distribution).

My question is: is there a provision of having multiple 'logical tables' in single instance of levelDB ?

From what I know, I can have multiple instances of levelDB running on my node each handling one table. But I do not want to do that, since in this case there will be serious contention (at disk I believe) when these multiple DB instances are accessed simultaneously. While having multiple logical tables in single instance of DB can give me advantages of levelDB optimizations for minimizing disk accesses.

Answer 1

If you want to have multiple "logical tables" in LevelDB, then you have to partition your key space or add a prefix to the keys. For each table create a different prefix, eg:

0x0001 is for table 1
0x0002 is for table 2
0x0003 is for table 3
and so on...

So a key would consist of the table prefix and the key itself: [0x0001,0xFF11] would address key 0xFF11 in table 1. You can then use a single LevelDB instance and have multiple "key spaces" which would correspond to "tables".

Answer 2

Your best option is partitioning the key space using a key prefix as suggested by Lirik. Though opening multiple databases is possible, I would not recommend it for your use case, since the databases will not share any buffers and caches. Working with multiple open databases may negatively impact performance, and it will make optimizing resource use (mostly memory) a lot harder.