correct convention call to a class method

Question

I cannot use __thiscall giving me the following error:
error C4234: nonstandard extension used : '__thiscall' keyword reserved for future use

I'm calling a class function from a dll:

typedef void(*SETDIRPROC)(void*,D3DXVECTOR3&);

void ZMyCharacter_SetDirection_Rev( void )
{
    if( pZMyCharacter == 0 )
    {
        printf( "cannot set direction now\n" );
        return;
    }
    SETDIRPROC SetDir_Proc = (SETDIRPROC)0x004A2AF0;
    D3DXVECTOR3 pos = D3DXVECTOR3(4.0f,2.0f,1.0f);
    SetDir_Proc( pZMyCharacter, pos );
}

pZMyCharacter is a void* pointer to the real class ZMyCharacter on the real application. It works, but I get debug errors (which can be ignored) warning that the call convention is different. It indeed is, since SETDIRPROC is __cdecl by default and I can't change it to __thiscall.

typedef void(__thiscall *SETDIRPROC)(void*,D3DXVECTOR3&); //error C4234

how do I get around with this?

Answer 1

I am going to take you down the dark and terrifying road to the land of undefined behavior...

The problem here is that you need to call pointer to a member function without actually having a pointer to a member function. You can use a bit of dark magic from the land of UB to accomplish this. This dark magic will let you convert a simple integer value into a fully usable pointer to member function. To do this we can create a union...

// Magical beans. Replace with your own beans if you have them.
struct ZMyCharacter {};

// Here be dark magic!
union FunctionAddress
{
    typedef void (ZMyCharacter::*MEMBER_FUNC)(D3DXVECTOR3);

    FunctionAddress(uintptr_t addr) : address(reinterpret_cast<void*>(addr)) {}

    MEMBER_FUNC function;
private:
    void* address;
};

This magical union will allow you to set pointer to member function from a simple integer value via its constructor...

FunctionAddress pZMyFunction(0x004A2AF0);

To use this you will need to make a couple of changes to ZMyCharacter_SetDirection_Rev to call the member function directly instead of passing it as a pointer to another function (which based on your previous question I assume is some inline assembly)...

void ZMyCharacter_SetDirection_Rev( void )
{
    if( pZMyCharacter == NULL )
    {
        printf( "cannot set direction now\n" );
        return;
    }
    D3DXVECTOR3 pos = D3DXVECTOR3(4.0f,2.0f,1.0f);

    // directly call using pointer to member function
    (pZMyCharacter->*pZMyFunction.function)( pos );
}

Keep in mind that if the function is virtual you are sidestepping the virtual function dispatch. If this is the case you should be prepared for situations where it breaks polymorphism. If you want to fully support the dispatch of virtual functions you will need to completely reverse engineer the layout of the virtual table. For this you can do something even simpler that doesn't use much dark magic.

// reverse engineered virutal function layour/ordering
struct ZMyCharacter
{
    virtual void func1();
    virtual void func2();
    virtual void func3();
    virtual void target_function(D3DXVECTOR3&);
    virtual void func4();
};

void ZMyCharacter_SetDirection_Rev( void )
{
    if( pZMyCharacter == NULL )
    {
        printf( "cannot set direction now\n" );
        return;
    }
    D3DXVECTOR3 pos = D3DXVECTOR3(4.0f,2.0f,1.0f);
    pZMyCharacter->target_function( pos );
}

I realize this changes a few things for your current code base. You should be able to integrate either the code or the concepts into what you are trying to accomplish.

---

The undefined behavior I mention is in regards to accessing a non-active member of a union. Typically this is considered bad.

Answer 2

I think it should look like this:

typedef void(__thiscall &ZMyCharacter::*SETDIRPROC)(D3DXVECTOR3&);
SETDIRPROC SetDir_Proc = (SETDIRPROC)0x004A2AF0;
static_cast<ZMyCharacter*>(pZMyCharacter)->SetDir_Proc( pos );

__thiscall is meant to be used on a member function pointer, not a free function like you had declared. The above should be closer to what the compiler needs--a member function type cast and called.

Answer 3

This is untested, but should give you a start. Note that it relies on MSVC implementation details, so it's not standard and won't work in other compilers:

#pragma warning(push)
#pragma warning(disable: 4608)
template < typename Src, typename Dest >
Dest force_cast( Src src )
{
union _convertor { Dest d; Src s; _convertor() : d(0), s(0) {} } convertor;
convertor.s = src;
return convertor.d;
}
#pragma warning(pop)

void func(ZMyCharacter* pZMyCharacter)
{
typedef void (ZMyCharacter::*F_SetDirection)(D3DXVECTOR3&);
F_SetDirection pfSetDirection = force_cast< FARPROC, F_SetDirection >(0x004A2AF0);
D3DXVECTOR3 pos = D3DXVECTOR3(4.0f,2.0f,1.0f);
(pZMyCharacter->*pfSetDirection)(pos);
}

Answer 4

This little (obscure) snippet here is not recommended at all, but as you can imagine, it works:

class ZMyCharacterHook
{
public:
    virtual void SetDir(D3DXVECTOR3&);
};

typedef void (*SETDIRPROC)(D3DXVECTOR3&);
typedef void (ZMyCharacterHook::*SETDIR_METHOD)(D3DXVECTOR3&);

SETDIRPROC SetDir_Proc = (SETDIRPROC)0x004A2AF0;

D3DXVECTOR3 pos = D3DXVECTOR3(4.0f, 2.0f, 1.0f);
((ZMyCharacterHook *)pZMyCharacter->*(SETDIR_METHOD &)SetDir_Proc)(pos);

It is not maintainable in any way, and should only be used if you really want to take the risk, as it has undefined behavior (take a look at the reference casting workaround).

If you want to be more discrete, but still against any maintainability and portability, you could use x86 MSVC inline assembly to explicit set ecx (which holds the this pointer when using thiscall), and then call a "simple" function by its address:

__asm
{
    pusha                    // save all registers
    mov  ecx, pZMyCharacter  // "this" pointer
    lea  eax, pos            // eax = &pos (parameter passed by reference)
    push eax                 // parameters pushed onto the stack
    mov  eax, 0x004A2AF0
    call eax                 // the function call itself
    popa                     // restore registers, no messed up code from here on
}