How to process text streams with \r correctly? I'd like some line buffered way, using Qt

Question

I'm using Qt and QProcess to read some data from other tools and printing them on my app. Think of it being a "terminal", for example.

I'm processing data using QProcess::canReadLine() and QProcess:readLine(), and that's wonderful. But some tools use \r to print progress bars on screen, and that's screwing with my parser. Since there is never some line to be read, my app just wait until the process finishes to print the last line: many lines glued together with \r instead of \n.

Anyways, is there someway to tell QProcess to use \r as linebreak also? I thought of implementing my QIODevice subclass, but I'd need to reimplement QProcess too, so that seems to be not the optimal approach.

I thought of using a middle buffer, and use this buffer to signal "hasLine" to my main program. I'd use QProcess::readyRead to populate the buffer, and then the buffer to populate my main app, but I'd like to just tell Qt that a \r is also OK as a linebreak. Is that possible?

Answer 1

I don't think it's possible to directly tell Qt to use '\r' as a linebreak. I thought that QTextStream could do that, but looking at its sources right now it seems to me that I was wrong.

One funny way of doing it would be to implement a custom QIODevice subclass that reads from another QIODevice and just replaces all '\r's with '\n's, delegating all other methods excep read() varieties to the original device. Then readLine() and QTextStream would work on the resulting stream just fine, I think. You'd have to deal somehow with the possible '\r\n' sequence, though. The upside is that you don't have to do any buffering in that class.

Something along these lines:

class CRFilter: public QIODevice {
    Q_OBJECT
public:
    CRFilter(QIODevice *device);
protected:
    virtual qint64 readData(char *data, qint64 maxSize);
    virtual qint64 writeData(const char *data, qint64 maxSize);
private:
    QIODevice *device;
};

CRFilter::CRFilter(QIODevice *device):
device(device)
{
    // delegate the readyRead() signal to this object
    connect(device, SIGNAL(readyRead()), SIGNAL(readyRead()));
    // and maybe other signals like bytesWritten() too...
}

qint64 CRFilter::readData(char *data, qint64 maxSize)
{
    qint64 res = device->read(data, maxSize);
    for (qint64 i = 0; i < res; i++) {
        if (data[i] == '\r')
            data[i] = '\n';
    }
    return res;
}

qint64 CRFilter::writeData(const char *data, qint64 maxSize)
{
    return device->write(data, maxSize);
}

Then you just do this:

QProcess process; // use QProcess methods on this
CRFilter reader(&p); // use QIODevice methods on this
reader.open(QIODevice::ReadWrite); // need this to convince read()/write() methods to work

I hadn't actually tested it, so it probably needs some debugging to get it right. I also think it's a bit ugly, but can't think of any really elegant solution.

Answer 2

Since I'm not using polymorphism with this, no problem inheriting publicly and overriding some methods and signals:

QCLIProcess.h

#ifndef QCLIPROCESS_H
#define QCLIPROCESS_H

#include <QProcess>

class QCLIProcess : public QProcess
{
    Q_OBJECT

public:
    explicit QCLIProcess(QObject *parent = 0);
    bool canReadLine() const;
    QString readLine();

signals:
    void readyRead();

private slots:
    void processLine();

private:
    QByteArray buffer;
    QStringList lines;
};

#endif // QCLIPROCESS_H

QCLIProcess.cpp

#include "QCLIProcess.h"
#include <QtCore>

QCLIProcess::QCLIProcess(QObject *parent) :
    QProcess(parent)
{
    setReadChannelMode(QProcess::MergedChannels);
    connect((QProcess *)this, SIGNAL(readyRead()), this, SLOT(processLine()));
}

void QCLIProcess::processLine(){
    buffer.append(readAll());

    int last = 0;
    for(int i=0; i<buffer.size(); i++){
        if (buffer.at(i) == '\n' || buffer.at(i) == '\r'){
            QString line(buffer.mid(last, i-last));
            line.append('\n');
            if (!line.isEmpty()) lines << line;
            last = i+1;
        }
    }
    buffer.remove(0, last);
    emit readyRead();
}

bool QCLIProcess::canReadLine() const {
    return !lines.isEmpty();
}

QString QCLIProcess::readLine(){
    QString line;

    if (!lines.isEmpty()){
        line = lines.at(0);
        lines.removeFirst();
    }

    return line;
}

UPDATE: I ended encapsulating the QProcess in a new class, rather than deriving it. This way I could control which signals and which slots I want to expose.

QLineBufferedCRFilteredProcess.h #ifndef QCLIPROCESS_H #define QCLIPROCESS_H

#include <QProcess>

class QLineBufferedCRFilteredProcess : public QObject
{
    Q_OBJECT

public:
    explicit QLineBufferedCRFilteredProcess(QObject *parent = 0);
    bool canReadLine() const;
    QString readLine();

    void start(const QString &program, const QStringList &arguments);
    void close();

signals:
    void readyRead();
    void finished(int exitCode, QProcess::ExitStatus exitStatus);

private slots:
    void processLine();

private:
    QProcess process;
    QByteArray buffer;
    QStringList lines;
};

#endif // QCLIPROCESS_H

QLineBufferedCRFilteredProcess.cpp #include "QLineBufferedCRFilteredProcess.h" #include

QLineBufferedCRFilteredProcess::QLineBufferedCRFilteredProcess(QObject *parent) :
    QObject(parent)
{
    process.setReadChannelMode(QProcess::MergedChannels);
    connect(&process, SIGNAL(readyRead()), SLOT(processLine()));
    connect(&process, SIGNAL(finished(int,QProcess::ExitStatus)), SIGNAL(finished(int,QProcess::ExitStatus)));
}

void QLineBufferedCRFilteredProcess::processLine()
{
    buffer.append(process.readAll());

    int last = 0;
    for(int i=0; i<buffer.size(); i++){
        if (buffer.at(i) == '\n' || buffer.at(i) == '\r'){
            QString line(buffer.mid(last, i-last));
            line.append('\n');
            if (!line.isEmpty()) lines << line;
            last = i+1;
        }
    }
    buffer.remove(0, last);
    emit readyRead();
}

bool QLineBufferedCRFilteredProcess::canReadLine() const
{
    return !lines.isEmpty();
}

QString QLineBufferedCRFilteredProcess::readLine()
{
    QString line;

    if (!lines.isEmpty()){
        line = lines.at(0);
        lines.removeFirst();
    }

    return line;
}

void QLineBufferedCRFilteredProcess::start(const QString &program, const QStringList &arguments)
{
    process.start(program, arguments);
}

void QLineBufferedCRFilteredProcess::close()
{
    process.close();
}