You are using the wide version of GetModuleHandle
(i.e. GetModuleHandleW
) thus you must pass it a valid wide string. Your mistake is that you are casting a non-wide string into a wide string which won't work. Use the following instead:
(DWORD)GetModuleHandleW(L"target.exe");
Or, the following, which accomplishes the same thing:
(DWORD)GetModuleHandleA("target.exe");