Have the monetdb's developers tested any other compression algorithm on it?

Question

Have the MonetDb's developers tested any other compression algorithm on it before?

Perhaps they have tested other compression algorithms ,but it's really had a negative performance impact.

So why haven't they improved this database's compression performance?

I am a student from China. MonetDb is really interesting me and I want to try to improve its compression performance.

So, I should make sure that any body have done this before.

It would be my grateful if you could answer my question.

That is because i really need this.

Thank you So much.

Answer 1

MonetDB only compresses String (Varchar and char) types using dictionary compression and only if the number of unique strings in a column is small.

Integrating any other kind of compression (even simple ones like Prefix-Coding, Run-length Encoding, Delta-compression, ...) need a complete rewrite of the system because the operators have to be made compression-aware (which pretty much means creating a new operator).

The only thing that may be feasible without a complete rewrite is having dedicated compression operators the compress/decompress data instead of spilling to disk. However, this would be very close to the memory compression apple implemented in Mavericks

Answer 2

MonetDB compresses columns using PFor compression. See http://paperhub.s3.amazonaws.com/7558905a56f370848a04fa349dd8bb9d.pdf for details. This also answers the your question about checking other compression methods.

The choice for PFOR is because of the way modern CPU's work, but really any algorithm that doesn't work with branches but with (only) arithmetics will do just fine. I've hit similar speeds with arithmetic coding in the past.