Unmanaged DLL doesn't work in Windows 7

Question

Last week I stumbled upon a problem and I'm not sure how to solve it. I already spent a whole day to try to solve it alone.

I have an unmanaged DLL that worked fine in my C# code on a Windows XP (32 bit) System. When I try to run this C# code on a newer System (for example Windows 7 (64 bit) it doesn't work anymore.

This is the code

[DllImport("MyTest.dll")]
[return: MarshalAs(UnmanagedType.LPStr)]
public static extern String DoSomething();

As I said this works fine on my Windows XP System but doesn't work on my Windows 7 System. When I try to run this code on my Windows 7 System I get the following Exception (translated into English)

The DLL "MyTest.dll": Recurrence too deep, Stackoverflow. (Exception from HRESULT: 0x800703E9) can not be loaded.

I'm guessing the problem is because of this:

Load 32bit DLL library in 64bit application

How to call win32 dll in windows 7

I'm just not sure about it because when I search for my exception in combination with DLLImport I can't find anything.

If this is really the problem what would be the best solution?

Answer 1

I solved the problem like this:

[DllImport("MyTest.dll", CharSet = CharSet.Ansi)]
private static extern IntPtr DoSomething();
public static string Do_Something()
{
   IntPtr tempPointer = DoSomething();
   string tempSomething = Marshal.PtrToStringAnsi(tempPointer);
   return tempSomething ;
}

The problem had to do with a corrupted heap. The handling of corrupted heaps is different in newer version of Windows and because of that my application crashed. It can be solved through changing the C#-Code or the C++-Code.

Detailed information about the problem can be found here: http://blogs.msdn.com/b/asiatech/archive/2009/12/24/net-application-may-crash-on-windows-2008-when-calling-function-from-native-c-dll.aspx

Answer 2

You could check:

1. unmanaged dll dependencies with dependency walker. There might be missing c++ redistributable. Look for MSVCR{xxx}.dll where {xxx} specifies version of the redistributable.
2. compilation settings of managed application that in the end loads unmanaged dll (Any CPU, x86 or x64). Might be that managed application is set to Any CPU but unmanaged application is x86 and therefore load of unmanaged dll fails.
3. installed .NET frameworks on windows xp and windows 7. The problem might be different patches applied on .NET frameworks but this is the least likely to be your problem.