WM_COPYDATA messages are not being handled

Question

I'm trying to communicate between two C# / .NET 3.5 applications using Windows Messages, but the messages I send appear to be received some of the time (but not all of the time) - why is this happening, and how do I ensure that the messages are properly handled all of the time. I have a client object as follows:

[DllImport("User32.dll", EntryPoint = "FindWindow")]
public static extern Int32 FindWindow(String lpClassName, String lpWindowName);

[DllImport("User32.dll", EntryPoint = "SendMessage")]
public static extern IntPtr SendMessage(IntPtr hWindow, int message, IntPtr wParam, IntPtr lParam);

public class WMTCPBridge
{

   private IntPtr TargetHwnd

   public void SendNumericMessage(Int32 messageCode,
     Int32 MessagePayload)
  {
     //for some reason String.fomat("blah 0x{0:X}",TargetHwnd) shows the number in decimal
     string sendNotice = String.Format("Sending to window 0x{0}", TargetHwnd.ToString("X")); 
     myTextOutput.Writeline(sendNotice);

     sendNotice = String.Format("Sending to window {0}", TargetHwnd);
     myTextOutput.Writeline(sendNotice);

     IntPtr unmanagedInt = Marshal.AllocHGlobal(sizeof(Int32));
     Marshal.WriteInt32(unmanagedInt,MessagePayload);
     IntPtr result = IntPtr.Zero;
     try
     {
        result = SendMessage(TargetHwnd, WM_COPYDATA, (IntPtr)messageCode,
           unmanagedInt);
     }
     finally
     {
        Marshal.FreeHGlobal(unmanagedInt);
     }
     myTextOutput.Writeline("Result is " + result);
     if ((int)result == 0)
     {
        myTextOutput.Writeline("Error code : " + GetThreadError());
     }
  }

public void GetTargetHandle(string targetName)
  {
     TargetHwnd = (IntPtr)FindWindow(null, targetName);
     if (TargetHwnd == null)
     {
        myTextOutput.Writeline("Could not connect to UI");
     }
     else
     {
        String outputLine = string.Format("Connected to window number 0x{0}", TargetHwnd.ToString("X"));
        myTextOutput.Writeline(outputLine);
        outputLine = string.Format("Connected to window number {0}", TargetHwnd);
        myTextOutput.Writeline(outputLine);
     }
  }
}

The main form of my test application owns an object of type WMTCPBridge, begins communication by calling GetTargetHandle and sends individual messages by calling the SendNumericMessage method. The server is a test harness that stands in for an existing application that I would like to avoid unneccessary changes to. It is this existing application that drives the choice of interface (I must use WM_COPYDATA, I must send a message type code via the wparam, if I want to send a single integer I should send the integer via the lparam instead of a Copydatastruct). The main form of the server application has the wndproc method overridden as follows:

  protected override void WndProc(ref Message m)
  {       
     Int32 messageCode=0;
     Int32 messagePayload=0;
     Debug.WriteLine(m);
     switch (m.Msg)
     {
        case WM_COPYDATA:
           {
              messageCode = (int)m.WParam;
              messagePayload = Marshal.ReadInt32(m.LParam);
              WriteLine("Received message with code " + messageCode +
                 " and payload " + messagePayload);
              break;
           }
        case WM_CLOSE:
           {
              WriteLine("Close blocked!");
              return;
              break;
           }
     }        
     base.WndProc(ref m);
  }

When I run the server and client together, the client reports that it is sending the messages to handle that I can see by Winspector is the server window's handle, the sendMessage function returns 0 and the application error is 0. Frequently, the server does not report getting any messages and Winspector does not show any WM_COPYDATA messages being sent to the server. However, if I keep sending messages from the client, some will be received by the server - I generally have streaks where either all messages get through or none do. When I modified the client to send WM_CLOSE messages, the server would inevitably receive them and close - even when I tried to trap the WM_CLOSE messages with the WndProc method as shown above.

What's happening to my messages? I'm especially confused because MSDN says that the SendMessage function only returns once a message has been processed.

Answer 1

You cannot ignore the fact that Windows wants LPARAM to point to a COPYDATASTRUCT structure. You however only allocate 4 bytes, not nearly enough to store that structure. What happens next is unpredictable, Windows will be reading past the memory that you allocated, looking for the value of COPYDATASTRUCT.cbData and lpData. You might get lucky and it reads cbData = 0. Or not so lucky and reading a non-zero value. Which will make it dereference lpData and that almost always generates an AccessViolation exception. You can tell when this happens, SendMessage() returns a value. One you didn't check so you don't know when this goes wrong.

As long as you want to keep using WM_COPYDATA, you have to provide it with proper arguments. The much better approach is to use named pipes or a socket. Which also avoid having to use FindWindow(), a very unreliable way to find a window handle.