Zip on-the-fly compression library in C for streaming

Question

Is there a library for creating zip files (the zip file format not gzip or any other compression format) on-the-fly (so I can start sending the file while it is compressing) for very large files (4 Gb and above).

The compression ratio does not matter much (mostly media files).

The library has to have a c-interface and work on Debian and OSX.

Answer 1

libarchive supports any format you want, on the fly and even in-memory files.

Answer 2

zlib supports compressing by chunks. you should be able to start sending a small chunk right after compressing it, while the library is still compressing the next chunk. (see this example)

(unfortunately, the file table is stored at the end of the zip file, so the file will be unusable until it is complete on the receiver side)

Answer 3

While this question is old and already answered I will note a new potential solution for those that find this.

I needed something very similar, a portable and very small library that created ZIP archives in a streaming fashion in C. Not finding anything that fit the bill I created one that uses zlib, available here:

https://github.com/CTrabant/fdzipstream

That code only depends on zlib and essentially provides a simple interface to creating ZIP archives. Most importantly (for me) the output can be streamed to a pipe, socket, whatever as the output stream does not need to be seek-able. The code is very small, a single source file and a header file. Works on OSX and Linux and probably elsewhere. Hope it helps someone beyond just me...