C# Marshal byte[] to COM SAFEARRAY parameter with “ref object” signature

Question

I've been going round and round in circles on Google on this, and I can find all kinds of discussion, lots of suggestions, but nothing seems to work. I have an ActiveX component which takes an image as a byte array. When I do a TLB import, it comes in with this signature:

int HandleImage([MarshalAs(UnmanagedType.Struct)] ref object Bitmap);

How do I pass a byte[] to that?

There's another function which can return the data with a similar signature, and it works because I can pass "null" in. The type that comes back is a byte[1..size] (non-zero bounded byte[]). But even if I try to pass in what came back, it still gets a type mismatch exception.

---

More details:

I've been editing the method in the IDispatch interface signature (using ILSpy to extract the interface from the auto-generated interop assembly). I've tried just about every combination of the following, it always gets Type mismatch exception:

1. Adding and removing the "ref"
2. Changing the parameter datatype to "byte[]" or "Array"
3. Marshalling as [MarshalAs(UnmanagedType.SafeArray, SafeArraySubType = VarEnum.VT_UI1)]. After playing around with MarshalAs quite a bit, I'm becoming convinced that IDispatch does not use those attributes.

Also tried using the "ref object" interface as is, and passing it different types: byte[], Array.CreateInstance(typeof(byte) (which I think are both identical, but I found someone suggesting it, so it couldn't hurt to try).

Here's an example of Delphi code that creates a proper array to pass in:

var
   image: OLEVariant;
   buf: Pointer;

image := VarArrayCreate([0, Stream.Size], VarByte);
Buf  := VarArrayLock(image);
Stream.ReadBuffer(Buf^, Stream.Size);
VarArrayUnlock(image);

Here's the C code to do the same thing. I guess if I can't get it to work from C#, I can invoke it through managed C++, although I'd rather have everything in one project:

long HandleImage(unsigned char* Bitmap, int Length)
{
    VARIANT vBitmap;
    VariantInit (&vBitmap);
    VariantClear(&vBitmap);

    SAFEARRAYBOUND bounds[1];
    bounds[0].cElements = Length;
    bounds[0].lLbound = 1;

    SAFEARRAY* arr = SafeArrayCreate(VT_UI1, 1, bounds);
    SafeArrayLock(arr);
    memcpy(arr->pvData, Bitmap, Length);
    SafeArrayUnlock(arr);
    vBitmap.parray = arr;
    vBitmap.vt = VT_ARRAY | VT_UI1;

    long result;
    static BYTE parms[] = VTS_PVARIANT;
    InvokeHelper(0x5e, DISPATCH_METHOD, VT_I4, (void*)&result, parms,
        &vBitmap);

    SafeArrayDestroy(arr);
    VariantClear(&vBitmap);

    return result;
}

Answer 1

I finally figured out how to do it in 100% C# code. Apparently Microsoft never considered the idea that someone might use a method with this signature to pass data in, since it marshals correctly going the other direction (it properly comes back as a byte[]).

Also, ICustomMarshaler doesn't get called on IDispatch calls, it never hit the breakpoints in the custom marshaler (except the static method to get an instance of it).

The answer by Hans Passant in this question got me on the right track: Calling a member of IDispatch COM interface from C#

The copy of IDispatch there doesn't contain the "Invoke" method on IUnknown, but it can be added to the interface, using types in System.Runtime.InteropServices.ComTypes as appropriate: http://msdn.microsoft.com/en-us/library/windows/desktop/ms221479%28v=vs.85%29.aspx

That means you get 100% control over marshaling arguments. Microsoft doesn't expose an implementation of the VARIANT structure, so you have to define your own: http://limbioliong.wordpress.com/2011/09/19/defining-a-variant-structure-in-managed-code-part-2/

The input parameters of Invoke are a variant array, so you have to marshal those to an unmanaged array, and there's a variant output parameter.

So now that we have a variant, what should it contain? This is where automatic marshaling falls down. Instead of directly embedding a pointer to the SAFEARRAY, it needs a pointer to another variant, and that variant should point to the SAFEARRAY.

You can build SAFEARRAYs via P/Invoking these methods: http://msdn.microsoft.com/en-us/library/windows/desktop/ms221145%28v=vs.85%29.aspx

So main variant should be VT_VARIANT | VT_BYREF, it should point to another variant VT_UI8 | VT_ARRAY, and that should point to a SAFEARRAY generated via SafeArrayCreate(). That outermost variant should then be copied into a block of memory, and its IntPtr sent to the Invoke method.