How to debug memory related errors from using PInvokes in Silverlight

Question

After running my silverlight 5.0 application with several PInvokes in it for 5 minutes or so, I get the following error:

Attempted to read or write protected memory

Likely, I'm getting a memory leak somewhere.

The problem is how do I debug this? I don't get a stacktrace at all so I can't pinpoint the offending code exactly.

By commenting out code and rerunning application several times, I think I managed to track down the problem but I can't figure out the problem.

Potentially Offending Code:

private void InitUSBEvents()
{
    const string clsName = "SLUsbClass";
    const string wndName = "SLUsbWindow";

    Win32.WNDCLASSEX wndClassEx = new Win32.WNDCLASSEX();

    wndClassEx.cbSize = Marshal.SizeOf(wndClassEx);
    wndClassEx.lpszClassName = clsName;
    wndClassEx.lpfnWndProc = WndProc;

    rClassAtomValue = Win32.RegisterClassEx2(ref wndClassEx);

    windowHandle = Win32.CreateWindowEx2(0, rClassAtomValue, wndName, 0, 0, 0, 0, 0, IntPtr.Zero, IntPtr.Zero, IntPtr.Zero, IntPtr.Zero);

    Win32Usb.RegisterKeyboardUsbEvents(windowHandle);
}

[AllowReversePInvokeCalls]
private IntPtr WndProc(IntPtr hWnd, WM msg, IntPtr wParam, IntPtr lParam)
{
    switch (msg)
    {
        case WM.INPUT:
            //Console.WriteLine("Key Event");
            break;
        default:
            return Win32.DefWindowProc(hWnd, msg, wParam, lParam);
    }

    return IntPtr.Zero;
}

PInvoke Related Declarations:

public class Win32
{
    [DllImport("user32.dll", SetLastError = true, EntryPoint = "CreateWindowEx")]
    public static extern IntPtr CreateWindowEx(
       WindowStylesEx dwExStyle,
       string lpClassName,
       string lpWindowName,
       WindowStyles dwStyle,
       int x,
       int y,
       int nWidth,
       int nHeight,
       IntPtr hWndParent,
       IntPtr hMenu,
       IntPtr hInstance,
       IntPtr lpParam);

    // Create a window, but accept a atom value.  
    [DllImport("user32.dll", SetLastError = true, EntryPoint = "CreateWindowEx")]
    public static extern IntPtr CreateWindowEx2(
       WindowStylesEx dwExStyle,
       UInt16 lpClassName,
       string lpWindowName,
       WindowStyles dwStyle,
       int x,
       int y,
       int nWidth,
       int nHeight,
       IntPtr hWndParent,
       IntPtr hMenu,
       IntPtr hInstance,
       IntPtr lpParam);

    [DllImport("kernel32.dll", EntryPoint = "LocalAlloc")]
    internal static extern IntPtr LocalAlloc_NoSafeHandle(
        LocalMemoryFlags uFlags, IntPtr sizetdwBytes);

    [DllImport("kernel32.dll", SetLastError = true)]
    internal static extern IntPtr LocalFree(IntPtr hMem);

    [DllImport("user32.dll", SetLastError = true, EntryPoint = "RegisterClassEx")]
    public static extern UInt16 RegisterClassEx2([In] ref WNDCLASSEX lpwcx); 

    [DllImport("user32.dll")]
    [return: MarshalAs(UnmanagedType.U2)]
    public static extern short RegisterClassEx([In] ref WNDCLASSEX lpwcx);

    [DllImport("user32.dll")]
    public static extern IntPtr DefWindowProc(IntPtr hWnd, WM uMsg, IntPtr wParam, IntPtr lParam); 
  }

  public class Win32Usb
  {

    public static bool RegisterKeyboardUsbEvents(IntPtr hWnd)
    {
        //Create an array of all the raw input devices we want to 
        //listen to. In this case, only keyboard devices.
        //RIDEV_INPUTSINK determines that the window will continue
        //to receive messages even when it doesn't have the focus.
        RAWINPUTDEVICE[] rid = new RAWINPUTDEVICE[1];

        rid[0].usUsagePage = 0x01;
        rid[0].usUsage = 0x06;
        rid[0].dwFlags = RIDEV_INPUTSINK;
        rid[0].hwndTarget = hWnd;

        return RegisterRawInputDevices(rid, (uint)rid.Length, (uint)Marshal.SizeOf(rid[0]));
    }

    [StructLayout(LayoutKind.Explicit)]
    internal class RAWINPUT
    {
        [FieldOffset(0)]
        public RAWINPUTHEADER header;
        [FieldOffset(16)]
        public RAWMOUSE mouse;
        [FieldOffset(16)]
        public RAWKEYBOARD keyboard;
        [FieldOffset(16)]
        public RAWHID hid;
    }

            [StructLayout(LayoutKind.Sequential)]
    internal class RAWINPUTDEVICELIST
    {
        public IntPtr hDevice;
        [MarshalAs(UnmanagedType.U4)]
        public int dwType;
    }

    [StructLayout(LayoutKind.Sequential)]
    internal struct RAWINPUTDEVICE
    {
        [MarshalAs(UnmanagedType.U2)]
        public ushort usUsagePage;
        [MarshalAs(UnmanagedType.U2)]
        public ushort usUsage;
        [MarshalAs(UnmanagedType.U4)]
        public int dwFlags;
        public IntPtr hwndTarget;
    }
  }

I can't really find anything wrong with the PInvoke declarations either. What are some good strategies to debug PInvoke related errors on Silverlight? Or is there a way to force the stacktrace to output? (I've tried turning on all exceptions to be thrown in Debug->Exception)

Answer 1

The WndProc delegate that you are passing in the wndClassEx struct is being garbage collected. When you pass a delegate to unmanaged code that delegate will only be kept alive for the time of the call to the unmanaged function. Once the call ends there is no longer reference to it in your Silverlight application so the garbage collection considers it dead and will collect it.

This example of creating a window class in Silverlight caches the WndProc delegate in a static method so that it does not get garbage collected.

Also see this (older, but still valid) MSDN article on P/Invoke.

Answer 2

@shf301 is correct that you need to take steps to stop your delegate from being collected. I won't attempt to repeat his point here.

However, on top of that issue, your structs are incorrectly defined. They will work under 32 bit, but not under 64 bit. The rule of thumb with FieldOffset is that you only ever use it with an offset of 0. This rule is what allows you to write code that works on both 32 and 64 bit. In your code you used an offset of 16 which is fine for 32 bit, but not 64 bit. Define your structures following this pattern:

[StructLayout(LayoutKind.Explicit)]
struct RAWINPUTUNION
{
    [FieldOffset(0)]
    public RAWMOUSE mouse;
    [FieldOffset(0)]
    public RAWKEYBOARD keyboard;
    [FieldOffset(0)]
    public RAWHID hid;
}

[StructLayout(LayoutKind.Sequential)]
struct RAWINPUT
{
    public RAWINPUTHEADER header;       
    public RAWINPUTUNION data;
}