How to glfwSetKeyCallback for different classes?

Question 1

Things get much easier when you step back for a moment and don't think in classes at all. Also a class is a type and types don't really describe a specific state. I think what you actually mean are instances if a class with different internal state.

So your problem then becomes to associate a callback with your state. In your post you wrote the following incomplete callback signature:

keycallback(GLFWwindow *window, int key, int scancode, int action, int mods);

There is missing something, namely the type it returns. Which is void so the full signature is

void GLFWCALL keycallback(
    GLFWwindow *window,
    int key,
    int scancode,
    int action,
    int mods);

Where GLFWCALL is a not further specified macro that expands into additional type signature qualifiers. You don't have to care about it anymore, than that this is the only type signature you can validly pass as a callback. Now image you'd write something like

class FooState
{
    void GLFWCALL keycallback(
        GLFWwindow *window,
        int key,
        int scancode,
        int action,
        int mods);  
};

What would be the type signature of this? Well, when you actually implement it, you're writing it down

void GLFWCALL FooState::keycallback(
    GLFWwindow *window,
    int key,
    int scancode,
    int action,
    int mods);

Look at this additional FooState::. This is not just some addition to the name or a namespace. It's actually a type modifier. It's like as if you added another parameter to the function (which is what in fact happens), and that parameter is a reference or a pointer (a pointer in the case of C++) to the class instance. This pointer is usually called this. So if you were to look at this function through the eyes of a C compiler (which doesn't know classes) and GLFW is written in C, the signature in fact looks like

void GLFWCALL keycallback(
    FooState *this,
    GLFWwindow *window,
    int key,
    int scancode,
    int action,
    int mods);

And it's pretty obvious that this doesn't fit the needs of GLFW. So what you need is some kind of wrapper that gets this additional parameter there. And where do you get that additional parameter from? Well that would be another variable in your program you have to manage.

Now we can use static class members for this. Within the scope of a class static means a global variable shared by all instances of the class and functions that are within the namespace of the class, but don't work on instances of it (so from a C compiler's perspective they're just regular functions with no additional parameter).

First a base class that gives us a unified interface

// statebase.h
class StateBase
{
    virtual void keycallback(
        GLFWwindow *window,
        int key,
        int scancode,
        int action,
        int mods) = 0; /* purely abstract function */

    static StateBase *event_handling_instance;
    // technically setEventHandling should be finalized so that it doesn't
    // get overwritten by a descendant class.
    virtual void setEventHandling() { event_handling_instance = this; }

    static void GLFWCALL keycallback_dispatch(
        GLFWwindow *window,
        int key,
        int scancode,
        int action,
        int mods)
    {
        if(event_handling_instance)
            event_handling_instance->keycallback(window,key,scancode,action,mods);
    }    
};

// statebase.cpp
#include "statebase.h"
// funny thing that you have to omit `static` here. Learn about global scope
// type qualifiers to understand why.
StateBase * StateBase::event_handling_instance;

Now the purely abstract virtual function is the trick here: Whatever kind of class derived from StateBase is referenced by the event_handling_instance by use of a virtual dispatch it will end up being called with the function implementation of the class type of that instance.

So we can now declare FooState being derived from StateBase and implement the virtual callback function as usual (but omit that GLFWCALL macro)

class FooState : BaseState
{
    virtual void keycallback(
        GLFWwindow *window,
        int key,
        int scancode,
        int action,
        int mods);  
};
void FooState::keycallback(
    GLFWwindow *window,
    int key,
    int scancode,
    int action,
    int mods)
{
    /* ... */
}

To use this with GLFW register the static dispatcher as callback

glfwSetKeyCallback(window, StateBase::keycallback_dispatcher);

And to select a specific instance as active callback handler you call its inherited void StateBase::setEventHandling() function.

FooState state;
state.setEventHandling();

A few parting words of advice:

If you've not been familiar with these concepts of OOP in C++ and the intricate how types and instances interact you should not do any OOP programming, yet. OOP was once claimed to make things easier to understand, while in fact the shrewd ways it got implemented in C++ actually makes one brain hurt and proliferates very bad style.

Here's an exercise for you: Try to implement what you try to achieve without using OOP. Just use structs without member functions in them (so that they don't become classes in disguise), try to think up flat data structures and how to work with them. It's a really important skill which people who jump directly into OOP don't train a lot.

Question 2

What you need for this to work is known as a closure (provides the necessary context to reference non-local storage). You cannot access class members in a traditional C callback because there is no way to satisfy the calling convention of a C++ class function (namely the added requirement of a this pointer).

Most frameworks get around this by allowing you to supply some void * user-data that is either passed directly to your callback or that can be queried when the callback is triggered. You can cast this void * to anything you want, in this case a pointer to a class object, and then invoke a member function belonging to that object.

By the way, GLFWCALL should really add nothing to this discussion. That is a macro that defines the default calling convention for the platform. It will never help you call a member function in a C++, its primary purpose was to ease things like DLL imports/exports.

Question 3

What @andon-m-coleman said

    auto keyCallback = []( GLFWwindow* window, int key, int scancode, int action, int mods ) {
        auto me = (Window*)glfwGetWindowUserPointer( window );
        me->KeyCallback( window, key, scancode, action, mods );
    };
    glfwSetWindowUserPointer( window, this );
    glfwSetKeyCallback( window, keyCallback );

Question 4

Make a separate controller routine for GLFW; have it receive events and pass them to whatever state that currently is on top of the state stack. (I normally had this directly in the application main loop.)

Question 5

I came up with a solution for gluing classes to glfw callbacks. It's fairly flexible, but will only allow you to register a class once, without ability to re-register.

Wrap a static reference to our class in a function, as a static function variable. Take the reference as a parameter, set it, and return a lambda's pointer. Note that the lambda doesn't need to capture anything, because our class reference is static, but only accessible within the function:

void  (*build_framebuffer_callback(rendering::renderer& r))(GLFWwindow*, int, int)
{
    static rendering::renderer& renderer = r;

    void (*callback)(GLFWwindow*, int, int) = ([](GLFWwindow* window, int width, int height) {
        renderer.resize(width, height);
    });

    return callback;
}

The signature matches this glfw callback header:

void framebuffer_size_callback(GLFWwindow* window, int width, int height);

Then, you can do:

glfwSetFramebufferSizeCallback(window, build_framebuffer_callback(renderer));

The above is for my renderer class, but I think this would work with input as well. You would then create a function on your input class that allows the lambda to set whatever it needs to set.

How to glfwSetKeyCallback for different classes?

The actual question: