gzdeflate() and large amount of data

Question

I've been building a class to create ZIP files in PHP. An alternative to ZipArchive assuming it is not allowed in the server. Something to use with those free servers.

It is already sort of working, build the ZIP structures with PHP, and using gzdeflate() to generate the compressed data.

The problem is, gzdeflate() requires me to load the whole file in the memory, and I want the class to work limitated to 32MB of memory. Currently it is storing files bigger than 16MB with no compression at all.

I imagine I should make it compress data in blocks, 16MB by 16MB, but I don't know how to concatenate the result of two gzdeflate().

I've been testing it and it seems like it requires some math in the last 16-bits, sort of buff->last16bits = (buff->last16bits & newblock->first16bits) | 0xfffe, it works, but not for all samples...

Question: How to concatenate two DEFLATEd streams without decompressing it?

Answer 1

PHP stream filters are used to perform such tasks. stream_filter_append can be used while reading from or writing to streams. For example

    $fp = fopen($path, 'r');
    stream_filter_append($fp, 'zlib.deflate', STREAM_FILTER_READ);

Now fread will return you deflated data.

Answer 2

This may or may not help. It looks like gzwrite will allow you to write files without having them completely loaded in memory. This example from the PHP Manual page shows how you can compress a file using gzwrite and fopen.

http://us.php.net/manual/en/function.gzwrite.php

function gzcompressfile($source,$level=false){
    // $dest=$source.'.gz';
    $dest='php://stdout'; // This will stream the compressed data directly to the screen.
    $mode='wb'.$level;
    $error=false;
    if($fp_out=gzopen($dest,$mode)){
        if($fp_in=fopen($source,'rb')){
            while(!feof($fp_in))
                gzwrite($fp_out,fread($fp_in,1024*512));
            fclose($fp_in);
            }
          else $error=true;
        gzclose($fp_out);
        }
      else $error=true;
    if($error) return false;
      else return $dest;
}