How to access QWidget from other threads

Question

I have

struct MyWidget : QWidget {
    // non-GUI related stuff:
    int data;
    int doSth();
};

I need to access a MyWidget instance from another thread (i.e. not the main thread). Is there any way to do that safely? I understand that I cannot access GUI related functions because some backends (e.g. MacOSX/Cocoa) don't support that. However, I only need to access data or doSth() in this example. But from what I have understand, there is simply no way to guarantee the lifetime of the object - i.e. if the parent window with that widget closes, the MyWidget instance gets deleted.

Or is there a way to guarantee the lifetime? I guess QSharedPointer doesn't work because the QWidget does its lifetime handling internally, depending on the parent widget. QPointer of course also doesn't help because it is only weak and there is no locking mechanism.

My current workaround is basically:

int widget_doSth(QPointer<MyWidget> w) {
    int ret = -1;
    execInMainThread_sync([&]() {
        if(w)
            ret = w->doSth();
    });
    return ret;
}

(execInMainThread_sync works by using QMetaMethod::invoke to call a method in the main thread.)

However, that workaround doesn't work anymore for some specific reason (I will explain later why, but that doesn't matter here). Basically, I am not able to execute something in the main thread at that point (for some complicated deadlock reasons).

Another workaround I'm currently thinking about is to add a global mutex which will guard the MyWidget destructor, and in the destructor, I'm cleaning up other weak references to the MyWidget. Then, elsewhere, when I need to ensure the lifetime, I just lock that mutex.

---

The reason why my current workaround doesn't work anymore (and that is still a simplified version of the real situation):

• In MyWidget, the data is actually a PyObject*.

• In the main thread, some Python code gets called. (It's not really possible to avoid any Python code calls at all in the main thread in my app.) That Python code ends up doing some import, which is guarded by some Python-import-mutex (Python doesn't allow parallel imports.)

• In some other Python thread, some other import is called. That import now locks the Python-import-mutex. And while it's doing its thing, it does some GC cleanup at some point. That GC cleanup calls the traverse function of some object which holds that MyWidget. Thus, it must access the MyWidget. However, execInMainThread_sync (or equivalently working solutions) will deadlock because the main thread currently waits for the Python-import-lock.

Note: The Python global interpreter lock is not really the problem. Of course it gets unlocked before any execInMainThread_sync call. However, I cannot really check for any other potential Python/whatever locks. Esp. I am not allowed to just unlock the Python-import-lock -- it's there for a reason.

One solution you might think of is to really just avoid any Python code at all in the main thread. But that has a lot of drawbacks, e.g. it will be slow, complicated and ugly (the GUI basically only shows data from Python, so there need to be a huge proxy/wrapper around it all). And I think I still need to wait at some points for the Python data, so I just introduce the possible deadlock-situation at some other point.

Also, all the problems would just go away if I could access MyWidget safely from another thread. Introducing a global mutex is the much cleaner and shorter solution, compared to above.

Answer 1

My solution:

struct MyWidget : QWidget {
    // some non-GUI related stuff:
    int someData;
    virtual void doSth();   

    // We reset that in the destructor. When you hold its mutex-lock,
    // the ref is either NULL or a valid pointer to this MyWidget.
    struct LockedRef {
        boost::mutex mutex;
        MyWidget* ptr;      
        LockedRef(MyWidget& w) : ptr(&w) {}
        void reset() {
            boost::mutex::scoped_lock lock(mutex);
            ptr = NULL;
        }
    };  
    boost::shared_ptr<LockedRef> selfRef;

    struct WeakRef;

    struct ScopedRef {
        boost::shared_ptr<LockedRef> _ref;
        MyWidget* ptr;
        bool lock;
        ScopedRef(WeakRef& ref);
        ~ScopedRef();
        operator bool() { return ptr; }
        MyWidget* operator->() { return ptr; }
    };

    struct WeakRef {
        typedef boost::weak_ptr<LockedRef> Ref;
        Ref ref;
        WeakRef() {}
        WeakRef(MyWidget& w) { ref = w.selfRef; }
        ScopedRef scoped() { return ScopedRef(*this); }
    };

    MyWidget();
    ~MyWidget();
};



MyWidget::ScopedRef::ScopedRef(WeakRef& ref) : ptr(NULL), lock(true) {
   _ref = ref.ref.lock();
   if(_ref) {
       lock = (QThread::currentThread() == qApp->thread());
       if(lock) _ref->mutex.lock();
       ptr = _ref->ptr;
   }
}

MyWidget::ScopedRef::~ScopedRef() {
   if(_ref && lock)
       _ref->mutex.unlock();
}

MyWidget::~QtBaseWidget() {
    selfRef->reset();
    selfRef.reset();
}

MyWidget::MyWidget() {
    selfRef = boost::shared_ptr<LockedRef>(new LockedRef(*this));
}

Now, everywhere I need to pass around a MyWidget pointer, I'm using:

MyWidget::WeakRef widget;

And I can use it from another thread like this:

MyWidget::ScopedRef widgetRef(widget);
if(widgetRef)
    widgetRef->doSth();

This is safe. As long as ScopedRef exists, MyWidget cannot be deleted. It will block in its destructor. Or it is already deleted and ScopedRef::ptr == NULL.

Answer 2

You can use the signal/slot mechanism, but it can be tedious, if the number of GUI controls is large. I'd recommend a single signal and slot to control the gui. Send over a struct with all the info needed for updating the GUI.

void SomeWidget::updateGUISlot(struct Info const& info)
{
  firstControl->setText(info.text);
  secondControl->setValue(info.value);
}

You don't need to worry about emitting signals, if the recipient is deleted. This detail is handled by Qt. Alternatively, you can wait for your threads to exit, after exiting the GUI threads event loop. You'll need to register the struct with Qt.

EDIT:

From what I've read from your extended question, you're problems are related to communication between threads. Try pipes, (POSIX) message queues, sockets or POSIX signals instead of Qt signals for inter-thread communication.

Answer 3

Personally I don't like designs where GUI stuff (ie: A widget) has non-GUI related stuff... I think you should separate these two from each other. Qt needs to keep the GUI objects always on the main thread, but anything else (QObject derived) can be moved to a thread (QObject::moveToThread).

Answer 4

It seems that what you're explaining has nothing at all to do with widgets, Qt, or anything like that. It's a problem inherent to Python and its threading and the lock structure that doesn't make sense if you're multithreading. Python basically presumes that any object can be accessed from any thread. You'd have the same problem using any other toolkit. There may be a way of telling Python not to do that - I don't know enough about the cpython implementation's details, but that's where you'd need to look.

That GC cleanup calls the traverse function of some object which holds that MyWidget

That's your problem. You must ensure that such cross-thread GC cleanup can't happen. I have no idea how you'd go about it :(

My worry is that you've quietly and subtly shot yourself in the foot by using Python, in spite of everyone claiming that only C/C++ lets you do it at such a grand scale.