How to store a uint8_t* variable returned from a C++ function in c#?

Question

I am calling a C++ dll from my C# program. The DLL consists of several functions and I am able to call most of them except this one.

The C++ function is like as below:

 __declspec(dllexport) uint8_t* myHash(const char *filename)
    {
         uint8_t *hash = (unsigned char*)malloc(72*sizeof(uint8_t));
         //some processing on hash  
         return hash;
    }

As can be seen in the above code, the hash function stores a character array. I want to receive the values in my C# program but I am not able to do it.

My C# code is like as below:

 [DllImport("myHash.dll", CharSet = CharSet.Ansi)]
        public static extern IntPtr myHash(string filename);

    IntPtr ptr = myHash(fileA);
            char[] result = new char[72];
            Marshal.Copy(ptr, result, 0, 72);

Answer 1

The problem is that char in C# is a 16 bit character element. Your C++ code returns an array of 8 bit uint8_t values. You should switch to using a byte array instead.

[DllImport("myHash.dll", CallingConvention=CallingConvention.Cdecl,
    CharSet = CharSet.Ansi)]
public static extern IntPtr myHash(string filename);
....
IntPtr ptr = myHash(fileA);
byte[] result = new byte[72];
Marshal.Copy(ptr, result, 0, 72);

I specified a calling convention because, as written, your function is __cdecl. Perhaps you omitted something in the transcribing of the question, but the declaration above matches the unmanaged code in the question.

This function would be much better designed to allow the caller to allocate the buffer. That avoids you having to export a deallocator from the C++ code. I'd write the C++ like this:

__declspec(dllexport) int myHash(const char *filename, uint8_t* hash)
{
     // calculate hash and copy to the provided buffer
     return 0; // return value is an error code
}

And the corresponding C# code:

[DllImport("myHash.dll", CallingConvention=CallingConvention.Cdecl,
    CharSet = CharSet.Ansi)]
public static extern int myHash(string filename, byte[] hash);
....
byte[] hash = new byte[72];
int retval = myHash(fileA, hash);

This function hard-codes in its interface that the buffer is of length 72. That might be reasonable, but it might make sense to pass the length of the buffer too so that the unmanaged code can defend against buffer overruns.

Note that although you refer to the output of this function as a character array, the use of uint8_t* makes it seem more likely to be a byte array. If it really is a character array, then you can use Encoding.GetString() to convert into a string.