Qt Downloading large files

Question

I am trying to make a FileDownloader class for my application which is compatible with downloading large file to avoid high memory usage during download. I wrote the following function to write the file out during download using the QNetworkReply’s readyRead() signal:

void FileDownloader::writeReady(){
    if(file->open(QIODevice::ReadWrite | QIODevice::Append)){
        QDataStream fout(file);
        fout << reply->readAll();
    } else{
        qDebug(file->errorString().toLatin1());
    }
    file->close();
}

The problem with this that it’s not writing the file how it supposed to be and the file gets corrupted. However if I use the finished() signal instead of the readyRead() signal, it perfectly writes out the file (it has a plus 4 byte size mistake, but I can open the file and read all of it’s content perfectly).

How can I make this compatible with readyRead() signal and large files?

Answer 1

you're using writeReady(), you should listen for readReady() instead !

Not sure if this can help but, 1) file->close() should be inside the if(file->open() otherwise it has no sense in closing it, also 2) readAll() may return an empty QByteArray() , so I would test for contents before sending it to the stream, and also avoid a QDataStream , but just use : QByteArray data = reply->readAll(); file->write(data)

Answer 2

QDataStream provides a Qt-specific serialization of binary data to a QIODevice, and is meant to be used on "both sides" of the streaming process (i.e. data serialized to a QIODevice using QDataStream is meant to be subsequently read back into a process using a QDataStream, preferably using the same version of the binary protocol). QDataStream therefore adds information to the streams it writes in order to provide versioning data, ensuring that future versions of QDataStream are backwards compatible, and that's why you're seeing extra data when you write the data to a file.

In your case, you'll want to drop the use of QDataStream because you're writing raw binary data to a file. This can be accomplished by removing the data stream class above, and replacing it with a direct call to the QIODevice's write method:

if (file->write(reply->readAll()) == -1) {
   // some error occurred
}

You might also want to consider just leaving the file open instead of opening it each time you have data available. I have no hard numbers, but there is almost certainly undesired overhead opening it each time, especially when you know you'll be writing a lot of data to it.