compressed length of a string by boost::iostreams

Question

I have a string (of some fixed length), which I need to compress and then compare the compressed lengths (as a proxy for redundancy in the data or as a rough approximation to the Kolmogorov complexity). Currently, I am using boost::iostreams for compression, which seems working well. However, I don't know how to obtain the size of the compressed data. Can someone help, please?

The code snippet is

#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/filtering_stream.hpp>
#include <boost/iostreams/copy.hpp>
#include <boost/iostreams/filter/gzip.hpp>
#include <boost/iostreams/device/file_descriptor.hpp>
#include <boost/filesystem.hpp>
#include <string>
#include <sstream>

namespace io = boost::iostreams;

int main() {

  std::string memblock;

  std::cout << "Input the string to be compressed:";
  std::cin >> memblock;

  std::cout << memblock << std::endl;

  io::filtering_ostream out;
  out.push(io::gzip_compressor());
  out.push(io::file_descriptor_sink("test.gz"));
  out.write (memblock.c_str(), memblock.size());

  std::cout << out.size() << std::endl;

  return 0;

}

Answer 1

You can try adding boost::iostreams::counter to you chain between the compressor and sink and then calling it's characters() member to get number of bytes that went through it.

This works for me:

#include <boost/iostreams/filter/counter.hpp>

...

io::filtering_ostream out;
out.push(io::counter());
out.push(io::gzip_compressor());
out.push(io::counter());
out.push(io::file_descriptor_sink("test.gz"));
out.write (memblock.c_str(), memblock.size());
io::close(out); // Needed for flushing the data from compressor

std::cout << "Wrote " << out.component<io::counter>(0)->characters() << " bytes to compressor, "
    << "got " << out.component<io::counter>(2)->characters() << " bytes out of it." << std::endl;

Answer 2

I figured out yet another (and slightly slicker) way to achieve the compressed length of a string. I thought sharing it here, but basically it is simply passing the uncompressed string to a filtered buffer and copying the output back to a string:

template<typename T>
inline std::string compressIt(std::vector<T> s){

    std::stringstream uncompressed, compressed;
    for (typename std::vector<T>::iterator it = s.begin();
         it != s.end(); it++)
        uncompressed << *it;

    io::filtering_streambuf<io::input> o;
    o.push(io::gzip_compressor());
    o.push(uncompressed);
    io::copy(o, compressed);

    return compressed.str();
}

Later one can easily get the size of the compressed string as

compressIt(uncompressedString).size()

I feel it is better for it does not required me to create an output file as previously.

cheers, Nikhil

Answer 3

one other way would be

stream<array_source> input_stream(input_data,input_data_ize);
stream<array_sink> compressed_stream(compressed_data,alloc_compressed_size);  
filtering_istreambuf out;
out.push(gzip_compressor());
out.push(input_stream);
int compressed_size = copy(out,compressed_stream);
cout << "size of compressed_stream" << compressed_size << endl;