QT - QFile - slow read from file

Question

I want to calculate MD5 hash of the file.

If I use fast_md5 with native lib http://twmacinta.com/myjava/fast_md5.php, then Java read file from HDD with 90 MB/s...120 MB/s and calc time is 70 seconds.

If I use QT

  QCryptographicHash hash(QCryptographicHash::Md5);

    QFile in("C:/file.mkv");

    QFileInfo fileInfo("C:/file.mkv");
    qint64 imageSize = fileInfo.size();

    const int bufferSize = 1000000;

    char buf[bufferSize+5];
    int bytesRead;

    if (in.open(QIODevice::ReadOnly)) {

        while ( (bytesRead = in.read(buf, bufferSize)) > 0) {
//            imageSize -= bytesRead;
//            hash.addData(buf, bytesRead);

        }

    }
    else {
        qDebug() << "Failed to open device!";
    }

    in.close();
    qDebug() << hash.result().toHex();

then my program read file from HDD with speed 20...78 MB/s and calc time is 210 seconds.

Is it possible to accelerate the processing of MD5 Calc in QT? May need to increase the buffer from 1000000 to a larger value?

Answer 1

The best solution is

/*max bytes to read at once*/
static const qint64 CHUNK_SIZE = 1024;

/*init hash*/
QCryptographicHash hash(Sha1);

/*open file*/
QFile file("foo.bar");
if (!file.open(QIODevice::ReadOnly))
    return;

/*process file contents*/
QByteArray temp = file.read(CHUNK_SIZE);
while(!temp.isEmpty())
{
    hash.addData(temp);
    temp = file.read(CHUNK_SIZE);
}

/*finalize*/
const QByteArray res = hash.result();
qDebug("Hash is: %s", res.toHex());

This solution provides to me the best read speed (about 100-120 MB/s) and the best calc time - 51...75 secs.

From https://qt-project.org/forums/viewthread/41886