Will global hooks work across x86 and x64

Question

I was trying to embed everything into a single COM dll. Currently I hooked the WH_GETMESSAGE and WH_CBT as below:

BOOL TouchDetector::SetMessageHook(BOOL Install)
{
if (Install)
{
    return ((mHookMessage = ::SetWindowsHookEx(WH_GETMESSAGE, MessageHookProc, mDll, 0)) != NULL)
         && ((mHookWin = ::SetWindowsHookEx(WH_CBT, WinHookProc, mDll, 0)) != NULL);
}
else
{
    return UnhookWindowsHookEx(mHookMessage)
        && UnhookWindowsHookEx(mHookWin);
}
};

I also put the shared variables like this:

#pragma data_seg(".shared")
TouchDetector*   pTouch = nullptr;
HHOOK            mHookMessage = NULL;
HHOOK            mHookWin = NULL;
#pragma data_seg()
#pragma comment(linker,"/section:.shared,rws")

By attaching to explorer.exe I can see that hooks are working but not global. I also tried SetWinEventHook however the same result: only react to the window I created or explorer.exe.

The COM dll it self is x64, since explorer is x64. Could this be the problem?

What I was trying to archive is to update my app's configuration when the foreground window changed. I know that I can simply start another thread to keep an eye on it. But I don't like that, currently the program only react passively to user input or win callbacks.

Answer 1

From the SetWindowsHookEx documentation:

SetWindowsHookEx can be used to inject a DLL into another process. A 32-bit DLL cannot be injected into a 64-bit process, and a 64-bit DLL cannot be injected into a 32-bit process. If an application requires the use of hooks in other processes, it is required that a 32-bit application call SetWindowsHookEx to inject a 32-bit DLL into 32-bit processes, and a 64-bit application call SetWindowsHookEx to inject a 64-bit DLL into 64-bit processes. The 32-bit and 64-bit DLLs must have different names.

To hook all applications on the desktop of a 64-bit Windows installation, install a 32-bit global hook and a 64-bit global hook, each from appropriate processes, and be sure to keep pumping messages in the hooking application to avoid blocking normal functioning.

The global hooks are a shared resource, and installing one affects all applications in the same desktop as the calling thread.

So you'll need to create two versions of your DLL, one for a 32-bit hook and one for a 64-bit hook. Be careful if you later decide to add a WH_MOUSE, WH_KEYBOARD, WH_JOURNAL* or WH_SHELL global hook:

Be aware that the WH_MOUSE, WH_KEYBOARD, WH_JOURNAL*, WH_SHELL, and low-level hooks can be called on the thread that installed the hook rather than the thread processing the hook. For these hooks, it is possible that both the 32-bit and 64-bit hooks will be called if a 32-bit hook is ahead of a 64-bit hook in the hook chain.

But for your WH_GETMESSAGE and WH_CBT hooks, you should be fine with two versions of the same DLL.