Can I make zlib to fully inflate a known number of bytes from the beginning?

StackOverflow https://stackoverflow.com/questions/22956758

  •  30-06-2023
  •  | 
  •  

Question

I have a data stream I want to compress and decompress with zlib. The stream consists of N sequential blocks of varying size. The data are similar, so I want to concatenate them to achieve better compression and keep a separate index of sizes to split them apart later. But I don't have to always decompress all of them, only up to M blocks from the beginning. Can I do this? I.e. can I tell zlib to keep decompressing the stream until I get 123456 bytes worth of decompressed data? Is it that simple as just telling zlib the size of the buffer and waiting for inflate() to return Z_OK or I need to specify some of the FLUSH constants?

Usage scenario: This can be a set of updates for a database, the most recent is packed first, the oldest last. Assume I have updates from 0 (no db) to 5. I pack them in reversed order: 5, 4, 3, 2, 1, 0. When the database does not exist, I extract them all and process from the last one to the first, fully creating the database and applying all the updates. But if the database is already there and is at v3, I only extract and apply updates 5 and 4, i.e. the first two blocks. I can, of course, pack the blocks separately, but if I concatenate them I'll get better compression.

Was it helpful?

Solution

Yes. Just provide an output buffer of the desired size, and make compressed data available as input to inflate() until it fills the output.

Note that to decompress block k, you will need to first decompress blocks 1..k-1.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top