Luabind and coroutines

Question

I'm having some trouble understanding how to use coroutines properly with luabind. There's a templated function:

template<class Ret> 
Ret resume_function(object const& obj, ...)

Where (Ret) is supposed to contain the values passed to yield by Lua.

My current points of confusion are:

• What happens if the function returns rather than calling yield? Does resume_function return the function's return value?
• How are you supposed to use this function if you don't know ahead of time which (or how many) parameters will be passed to yield? For example, if there are multiple possible yielding functions the function may call.
• What is the type of Ret if multiple values are passed to yield?

Am I just entirely mistaken as to how all this works? I envision something like this. On the Lua side:

local img = loadImage("foo.png")

loadImage would be a C++ function which requests the image to be loaded in a different thread and then calls lua_yield, and some time later luabind::resume_function gets called with img as a parameter.

Should I pass "foo.png" to yield as a parameter? To a different function before I call yield, and then never pass any values to yield? What's the right way to structure this? I'm obviously misunderstanding something here.

Answer 1

Where (Ret) is supposed to contain the values passed to yield by Lua.

Luabind only supports single return values, so it only will return the first value passed to coroutine.yield.

What happens if the function returns rather than calling yield? Does resume_function return the function's return value?

Yes, you get its return value.

How are you supposed to use this function if you don't know ahead of time which (or how many) parameters will be passed to yield? For example, if there are multiple possible yielding functions the function may call.

That's up to you; they're your functions. You have to develop conventions about what the yielding function(s) receive as parameters, and what the function resuming the coroutine provides.

What is the type of Ret if multiple values are passed to yield?

Whatever you want it to be. It's the template parameter. The number of parameters to a function has no bearing on the return values that the function provides.

Remember: Lua functions take any number of parameters and can return anything. All Luabind can do is pass along the parameters you give it and convert the return value from Lua functions into what you expect that return value to be. Luabind will do type-checking on the return value of course. But it is your responsibility to make sure that the functions yielding/returning will return something that is convertable to the type the user provides for Ret.

loadImage would be a C++ function which requests the image to be loaded in a different thread and then calls lua_yield, and some time later luabind::resume_function gets called with img as a parameter.

If you're using Luabind, never call lua_yield directly. The proper way to yield in Luabind is to add an attribute to a function you register that will yield whenever you return from the function. The syntax is as follows:

module(L)
[
    def("do_thing_that_takes_time", &do_thing_that_takes_time, yield)
];

That is, a C++ function that yields must always yield. This is a limitation of Luabind, as with regular Lua, you can choose whether to yield or not as you see fit.

Also, don't forget that Lua coroutines are not the same thing as actual threads. They are not preemptive; they will only execute when you explicitly tell them to with coroutine.resume or an equivalent resume call.

Also, you should never run the same Lua instance from multiple C/C++ threads; Lua is not thread-safe within the same instance (which more or less means the same lua_State object).

What you seem to want to do is have Lua call some function in C++ that itself spawns a thread to do some process, then have the Lua code wait until that thread is complete and then receives its answer.

To do that, you need to give to the Lua script an object that represents the C++ thread. So your loadImage function should not be using coroutine logic; it should return an object that represents the C++ thread. The Lua script can ask the object if it has completed, and if it has, it can query data from it.

The place where coroutines can come into play here is if you don't want the Lua script to wait until this is finished. That is, you're calling the Lua script every so often, but if the C++ thread isn't done, then it should just return. In which case, you can do something like this:

function loadImageAsCoroutine(imageFilename)
    local cppThread = cpp.loadImage(imageFilename);

    local function threadFunc(cppThread)
        if(cppThread:isFinished()) then
            local data = cppThread:GetImage();
            return data;
        else
            coroutine.yield();
        end
    end

    local thread = coroutine.create(threadFunc);

    local errors, data = assert(coroutine.resume(thread, cppThread));

    if(coroutine.status(thread) == "dead") then
        return data;
    else
        return thread;
    end
end

This function returns a coroutine or the image data itself. The caller of this function should check the type; if the type is "thread", then the C++ thread hasn't finished yet. Otherwise, it is the image data.

The caller of this function can pump the coroutine however much they want with some equivalent of coroutine.resume (whether it's luabind::resume_function or whatever). Each time, check the return value. It will be nil if the C++ thread hasn't finished, and not nil otherwise.