Exporting c++ function, declared in vb6, getting "remote procedure call failed" runtime error

Question

I have a dll written in C++ that is being exported like so:

extern "C" {
    extern _declspec(dllexport) (VOID) Merge(LPCSTR szFileName, LPCSTR szMergeString, DWORD dwAlignment, BOOL fRepeat);
}

That function is defined elsewhere as:

extern _declspec(dllexport) (VOID) Merge(LPCSTR szFileName, LPCSTR szMergeString, DWORD dwAlignment, BOOL fRepeat)
{   
    Tools::Merge(gcnew String(szFileName), gcnew String(szMergeString), dwAlignment, fRepeat == TRUE);
}

In my vb6 dll, I have declared the remote call like this:

Private Declare Sub Merge Lib "Tools.dll" (fileName As String, mergeString As String, Alignment As Long, repeated As Boolean)

Then I call it in the code like this:

Merge strPageFileName, "COPY", 5, True

When I run the app, I get "Automation error: the remote procedure call failed" in a message box.

Tools.dll is in C:\Windows\system32. I tested with a bad dll name, to make sure that wasn't the problem, and get an error saying it could not find the file, so I know that's not it. Anything else I need to do to get this to work?

EDIT: Changing the variable type per xxbbcc's answer below allowed this to function correctly. However, it created a new, so far utterly unsolvable problem. I created another question here that has some of the details about it.

The remote call is happening in a COM+ dll, inside a function that returns type integer to another non-COM dll. For some strange reason, the COM+ dll's function is now returning a variant of type VT_ERROR instead of an integer. I have tested this thoroughly and determined that it does not matter what I do with the return value beforehand, setting it to any value, etc, it always returns an error. The odd part is that the value of the error is apparently 0. I have called CInt(returnValue), CStr(returnValue), and CLng(returnValue), and they return 0, Error 0, and 0, respectively.

I have been unable to find any information explaining why a function that is supposed to return an integer would suddenly be returning an error instead. This is giving me a type mismatch error in the non-COM+ function, as it tries to assign the return to a local integer. Changing that to a variant allowed me to do the casts above. Calling VarType(returnValue) gives 10, which is vbError. We don't want to potentially lose failure information from other calls inside the COM+ dll, so I can't just ignore the return value.

Answer 1

My current solution to this has been to write a tiny c++ console application that calls the exported function, and call the exe from within the vb6 dll.

Answer 2

BOOL should be defined as Long in VB6 and the value passed for it should be 0 or 1, not boolean values. It should also be ByVal - since parameters are passed by values in C / C++, you must make sure that your VB6 program follows that. Right now your VB6 program passes a pointer to a Boolean, rather than an actual Long. Your strings should also be defined as ByVal.

This article - while dated - has some useful information about why strings should be passed from VB6 as ByVal to C code.

Another thing - I never successfully exported a function to VB6 using declspec(dllexport). I did a lot of C / VB6 coding in the past but the only way it ever worked was through DEF files. I can't explain why (it may have been because of my ignorance). Ignore this part of my answer if fixing the BOOL parameter solves your issue. :)

Answer 3

Try declaring the String parameters in your VB6 Declare statement ByVal. Change "filename as String" to "ByVal filename as String", etc.