Does leveldb have structural problems if values are blank and there's just a key?

StackOverflow https://stackoverflow.com/questions/10762509

  •  11-06-2021
  •  | 
  •  

Question

I'm working on a design where secondary indexes to data will be built with all the information in the key, needing nothing in the value side. Is this likely to cause problems?

I'm not asking is it technically possible to have a blank value. Are there any structural consequences, for example: adding sorted keys can unbalance some tree structures? (I'm not saying leveldb uses trees, just trying to think of an analogy ;-) )

ie: say a "primary record" looks like (nulls as separators)

  • key = uniqueTableID \0 uniqueRowID
  • value = some collection of fields

a secondary index to a typical single-valued field would look like:

  • key = uniqueFieldID \0 keyValue \0 uniqueRowID

that allows iteration by the partial key [uniqueFieldID \0 keyValue] and it also makes it easy to find these keys and delete them if the main record is deleted or key value changes, working back from the main record's uniqueRowID. So there might be several key values ending in the same uniqueRowID but can only ever be one key for the particular combination starting with a uniqueFieldID and ending with a uniqueRowID

The only thing is that I don't have any need to put a value in the value side of the pair.

I'm pretty happy with this conceptual design, just checking to see if anyone can spot holes in it. For example, if it would distort leveldb internals causing performance issues.

I expect there would be tens of thousands of such keys in one particular app.

As an example of a value we might want to store, a secondary word index to a text field mightlook like:

  • key = uniqueFieldID \0 keyValue \0 GUID
  • value = count of word occurrences or maybe a list of offsets if scanning large blobs was expensive
Was it helpful?

Solution

The keys and values in LevelDB are opaque arrays and a quick look into the documentation for constructor of a Slice shows how you can create an empty slice:

// Create an empty slice.
Slice() : data_(""), size_(0)

This is useful exactly for the type of situations where you don't have any value data.

OTHER TIPS

It should be fine since even leveldb stores deletes as keys without values. Internally leveldb uses and prefix length encoding for keys in each SST which would help further in reducing the key sizes for your particular case. The only skewness in you case would be in the index sizes. Typically the index size would be a small fraction of the data blocks (assuming small keys and relatively big values), while in your case the index could be relatively big, since index stores a key per data block.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top