Managed C++ dll function throws access violation on return when called from Visual Studio process

Question

I have a managed C++ dll which is used as a EEAddIn for Visual Studio debugger. My dll exports one function accepting and returning unmanaged types. The declaration of the function in the dll is as follows (DEBUGHELPER is a plain structure):

extern "C" 
{
#pragma unmanaged
  __declspec(dllexport) HRESULT CvMatViewer(DWORD dwAddress, DEBUGHELPER *pHelper, 
    int nBase, BOOL bUniStrings, char *pResult, size_t max, DWORD reserved)
  {
    strcpy_s(pResult, max, "Value is processed");
    return S_OK;
  }
}

The purpose of my dll is to handle variables from Visual Studio debugger and to format them in arbitrary way prior to displaying in Watch Window (that what EEAddIn is for, pretty cool).

I have two Visual Studios launched - one for my managed dll and one for native debugged exe. I can attach my dll to the process of the Visual Studio running native exe and set a breakpoint. When user adds variable in the Watch window, my breakpoint is hit. I can see that all parameters is passed correctly and function works fine till the return statement is reached.

After that, I have an Unhandled exception at 0x51ea9de4 in devenv.exe: 0xC0000005: Access violation reading location 0x00000000.

The same works fine for unmanaged dlls. So can anyone guess where the problem hides?

Both Visual Studios are of 2010 version.

Yes, I can use unmanaged dll to accept this call and then pass control to another managed dll, but the Occam's razor principle tells me not to do that.

Update: pResult is not a NULL, and exception is thrown even if I comment strcpy_s out.

Answer 1

It looks like you have to set stdcall calling convention on this function explicitly, this should help.

__declspec(dllexport) HRESULT __stdcall CvMatViewer(DWORD dwAddress, DEBUGHELPER *pHelper, 
    int nBase, BOOL bUniStrings, char *pResult, size_t max, DWORD reserved);

Answer 2

I would check if pResult is actually not a null pointer before copying data in it. That is most likely you problem, given the size of the function.