Why CLRHosting API not working even if the GetLastError is 0?

Question

I am trying to load a managed C# dll into a managed C# process by following this tutorial. I have done a fair bit of coding in C/C++ and have working knowledge of MS COM, but C# and managed code is a completely new beast for me, so be forgiving if I am doing anything wrong. I have .NET 4.5 on my system and this is the same runtime being used by default (I think). The code so far (mostly copied from the above link):

Code - C#

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using System.Windows.Forms;

namespace InjectSample
{
    public class Program
    {
        int EntryPoint(String pwzArgument)
        {
            System.Media.SystemSounds.Beep.Play();

            MessageBox.Show(
                "I am a managed app.\n\n" +
                "I am running inside: [" +
                System.Diagnostics.Process.GetCurrentProcess().ProcessName +
                "]\n\n" + (String.IsNullOrEmpty(pwzArgument) ?
                "I was not given an argument" :
                "I was given this argument: [" + pwzArgument + "]"));

            return 0;
        }

        static void Main(string[] args)
        {
            Program prog = new Program();
            prog.EntryPoint("hello world");
        }
    }
}

Native Code

#include <metahost.h>
#pragma comment(lib, "mscoree.lib")
#import "mscorlib.tlb" raw_interfaces_only \
    high_property_prefixes("_get","_put","_putref") \
    rename("ReportEvent", "InteropServices_ReportEvent")


#include <strsafe.h>
void ErrorExit(LPCWSTR lpszFunction, DWORD dwLastError) 
{ 
    if(dwLastError == 0) return;
    LPVOID lpMsgBuf;
    LPVOID lpDisplayBuf;
    DWORD dwFlag = FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS;
    FormatMessage( dwFlag, NULL, dwLastError, MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), (LPTSTR) &lpMsgBuf, 0, NULL );

    lpDisplayBuf = (LPVOID)LocalAlloc(LMEM_ZEROINIT, (lstrlen((LPCTSTR)lpMsgBuf) + lstrlen((LPCTSTR)lpszFunction) + 40) * sizeof(TCHAR)); 
    StringCchPrintf((LPTSTR)lpDisplayBuf, LocalSize(lpDisplayBuf) / sizeof(TCHAR), TEXT("%s failed with error %d: %s"), lpszFunction, dwLastError, lpMsgBuf); 

    ::MessageBox(NULL, (LPCTSTR)lpDisplayBuf, TEXT("Error"), MB_OK); 

    LocalFree(lpMsgBuf);
    LocalFree(lpDisplayBuf);
}

int wmain(int argc, wchar_t* argv[])
{
    HRESULT hr;
    ICLRMetaHost *pMetaHost = NULL;
    ICLRRuntimeInfo *pRuntimeInfo = NULL;
    ICLRRuntimeHost *pClrRuntimeHost = NULL;

    // build runtime
    hr = CLRCreateInstance(CLSID_CLRMetaHost, IID_PPV_ARGS(&pMetaHost));
    hr = pMetaHost->GetRuntime(L"v4.0.30319", IID_PPV_ARGS(&pRuntimeInfo));
    hr = pRuntimeInfo->GetInterface(CLSID_CLRRuntimeHost, 
        IID_PPV_ARGS(&pClrRuntimeHost));

    // start runtime
    hr = pClrRuntimeHost->Start();

    // execute managed assembly
    DWORD pReturnValue;
    SetLastError(0);
    hr = pClrRuntimeHost->ExecuteInDefaultAppDomain(
        L"C:\\Temp\\InjectSample.exe", 
        L"InjectExample.Program", 
        L"int EntryPoint(String pwzArgument)", 
        L"hello .net runtime", 
        &pReturnValue);

    ErrorExit(L"ExecuteInDefaultAppDomain()", GetLastError());
    // free resources
    pMetaHost->Release();
    pRuntimeInfo->Release();
    pClrRuntimeHost->Release();

    return 0;
}

Problem

Now the problem is that when I execute the native code, GetLastError() returns 0. That is just after the call to ExecuteInDefaultAppDomain(). As per the Codeproject link, it should have shown the dialog box, but in my case it is not showing anything.

I am not sure about the problem, any suggestion/pointer will be helpful. Thanks.

Answer 1

The hosting api uses COM, it doesn't report errors through GetLastError(). The return value of the method is the error code. It is an HRESULT, the hosting api generally returns error codes like 0x8013xxxx. You'll find the xxxx values in the CorError.h SDK header.

You'll need something like this:

hr = pClrRuntimeHost->ExecuteInDefaultAppDomain(...);
if (FAILED(hr)) ErrorExit(L"Execute", hr);

And add this check to every call you make. Not doing this is likely to make your program crash in a hard to diagnose way. And you need all help you can get, you no longer have the friendly .NET exceptions to tell you what went wrong.

Other survival strategies are using mixed-mode debugging so you have a shot at diagnosing managed exceptions before they turn into an HRESULT, writing an AppDomain.CurrentDomain.UnhandledException event handler in your managed code, using IErrorInfo so you can get the exception message in your host, using the proper method name, it is L"EntryPoint", and making it static as required.