Using qCompress/qUncompress for multi-core

Question

Is it possible to utilize multicore for internal QT functions such as qCompress/qUncompress?

Thank you.

Answer 1

qCompress and qUncompress internally use zlib, and its algorithm is not easily parallellizable. The way other do it, like pigz for example, is to cut up the data into chunks that are then compressed in parallel. It requires a different file format than what zlib normally expects.

If you have control over your own data, you can split the data into multiple chunks and compress/decompress them in parallel. The number of chunks can be as large as the maximum number of threads envisioned for the decompression, but when you do compress/decompress, you only run QThread::idealThreadCount() number of threads in parallel, they may process more than one chunk each. The minimal size of the chunk has to make sense so as not to impact the compression ratio too much. You may need to experiment, but I'd gather that chunks below 128kb make little sense.