C++ Inflate gzip char array

Question

I'm attempting to use zlib to uncompress (inflate) some IP packet payload data that is compressed via gzip. However, I'm having some difficultly understanding some of the documentation provided by zlib that covers inflation. I have a char array that my program fills but I can't seem to inflate it with the following code:

const u_char payload; /*contains gzip data, 
                      captured prior to this point in the program*/

/*read compressed contents*/
int ret; //return val
z_stream stream;
unsigned char out[MEM_CHUNK]; //output array, MEM_CHUNK defined as 65535

/* allocate inflate state */
stream.zalloc = Z_NULL;
stream.zfree = Z_NULL;
stream.opaque = Z_NULL;
stream.avail_in = size_payload; // size of input
stream.next_in = (Bytef *)payload; // input char array
stream.avail_out = (uInt)sizeof(out); // size of output
stream.next_out = (Bytef *)out; // output char array

ret = inflateInit(&stream);
inflate(&stream, Z_NO_FLUSH);
inflateEnd(&stream);

printf("Inflate: %s\n\n", out);

In the zlib documentation, they have inflate continually called via a do/while loop, checking for the Z_STREAM_END flag. I'm a bit confused here, because it seems they're working from a file while I'm not. Do I need this loop as well, or am I able to provide a char array without looping over inflate?

Any guidance here would really be appreciated. I'm pretty new to both working with compression and C++.

Thanks.

Answer 1

Assuming you are giving inflate an appropriate and complete "compressed stream", and there is enough space to output the data, you would only need to call inflate once.

Edit: It is not written out as clearly as that in the zlib documentation, but it does say:

inflate decompresses as much data as possible, and stops when the input buffer becomes empty or the output buffer becomes full. It may introduce some output latency (reading input without producing any output) except when forced to flush.

Of course, for any stream that isn't already "in memory and complete", you want to run it block by block, since that's going to have less total runtime (you can decompress while the data is being received [from network or filesystem pre-fetch caching] for the next block).

Here's the whole function from your example code. I've removed the text components from the page to concentrate the code, and marked sections with letters // A , // B etc, then marked tried to explain the sections below.

int inf(FILE *source, FILE *dest)
{
    int ret;
    unsigned have;
    z_stream strm;
    unsigned char in[CHUNK];     // A
    unsigned char out[CHUNK];

    /* allocate inflate state */
    strm.zalloc = Z_NULL;        // B
    strm.zfree = Z_NULL;
    strm.opaque = Z_NULL;
    strm.avail_in = 0;
    strm.next_in = Z_NULL;
    ret = inflateInit(&strm);    // C
    if (ret != Z_OK)
        return ret;

    /* decompress until deflate stream ends or end of file */
    do {
        strm.avail_in = fread(in, 1, CHUNK, source);     // D
        if (ferror(source)) {
            (void)inflateEnd(&strm);      // E
            return Z_ERRNO;
        }
        if (strm.avail_in == 0)           // F
            break;
        strm.next_in = in;                // G


        /* run inflate() on input until output buffer not full */
        do {
            strm.avail_out = CHUNK;       // H
            strm.next_out = out;

            ret = inflate(&strm, Z_NO_FLUSH);  // I
            assert(ret != Z_STREAM_ERROR);  /* state not clobbered */
            switch (ret) {
            case Z_NEED_DICT:
                ret = Z_DATA_ERROR;     /* and fall through */
            case Z_DATA_ERROR:
            case Z_MEM_ERROR:
                (void)inflateEnd(&strm);
                return ret;
            }

            have = CHUNK - strm.avail_out;     // J
            if (fwrite(out, 1, have, dest) != have || ferror(dest)) {
                (void)inflateEnd(&strm);    
                return Z_ERRNO;
            }

        } while (strm.avail_out == 0);         // K

        /* done when inflate() says it's done */
    } while (ret != Z_STREAM_END);             // L

    /* clean up and return */
    (void)inflateEnd(&strm);
    return ret == Z_STREAM_END ? Z_OK : Z_DATA_ERROR;
}

A: in is the input buffer (we read from a file into this buffer, then pass it to inflate a while later. out is the output buffer, which is used by inflate to store the output data.

B: Set up a z_stream object called strm. This holds various fields, most of which are not important here (thus set to Z_NULL). The important ones are the avail_in and next_in as well as avail_out and next_out (which are set later).

C: Start inflation process. This sets up some internal data structures and just makes the inflate function itself "ready to run".

D: Read a "CHUNK" amount of data from file. Store the number of bytes read in strm.avail_in, and the actual data goes into in.

E: If we errored out, finish the inflate by calling inflateEnd. Job done.

F: No data available, we're finished.

G: Set where our data is coming from (next_in is set to the input buffer, in).

H: We're now in the loop to inflate things. Here we set the output buffer up: next_out and avail_out indicate where the output goes and how much space there is, respectively.

I: Call inflate itself. This will uncompress a portion of the input buffer, until the output is full.

J: Calculate how much data is available in this step (have is the number of bytes).

K: Until we have space left when inflate finished - this indicates the output is completed for the data in the in buffer, rather than out of space in the out buffer. So time to read some more data from the input file.

L: If the error code from the inflate call is "happy", go round again.

Now, obviously, if you are reading from a network, and uncompressing into memory, you need to replace the fread and fwrite with some suitable read from network and memcpy type calls instead. I can't tell you EXACTLY what those are, since you haven't provided anything to explain where your data comes from - are you calling recv or read or WSARecv, or something else? - and where is it going to?