Callback from C# COM dll to Delphi app causes memory leak

Question

I have a COM server written in C#, and a COM client written in Delphi. I've implemented a call-back mechanism that is simple and elegant and it works like a charm. However, FastMM4 reports that my Delphi client is creating a memory leak. I've distilled the application to the essence of where the leak is coming from. I've the leak is caused by the way that the object is being reference counted (it never goes to zero so never gets destroyed), so I'm trying to understand why the reference counting is working the way that it is, and is it because of something I'm doing wrong in my implementation.

I've cut the code down as much as I can, but it still seems like a lot to include in a question. But I really don't know how else to explain what I'm doing. I have the two projects (C# and Delphi) wrapped up nice and tidy in a zip file, but it doesn't seem like I can attach that anywhere.

I'm declaring two interfaces on the C# side (ICOMCallbackContainer and ICOMCallbackTestServer) and implementing one of them there (COMCallbackTestServer). I'm implementing the other interface on the Delphi side (TCOMCallbackContainer) and passing the Delphi class to the C# class.

This is the C# COM Server:

namespace COMCallbackTest
{
    [ComVisible(true)]
    [Guid("2AB7E954-0AAF-4CFE-844C-756E50FE6360")]
    public interface ICOMCallbackContainer
    {
        void Callback(string message);
    }

    [ComVisible(true)]
    [Guid("7717D7AE-B763-48BC-BA0B-0F3525BEE8A4")]
    public interface ICOMCallbackTestServer
    {
        ICOMCallbackContainer CallbackContainer { get; set; }
        void RunCOMProcess();
        void Dispose();
    }

    [ComVisible(true)]
    [Guid("CF33E3A7-0886-4A0D-A740-537D0640C641")]
    public class COMCallbackTestServer : ICOMCallbackTestServer
    {
        ICOMCallbackContainer _callbackContainer;

        ICOMCallbackContainer ICOMCallbackTestServer.CallbackContainer
        {
            get { return _callbackContainer; }
            set { _callbackContainer = value; }
        }

        void ICOMCallbackTestServer.RunCOMProcess()
        {
            if (_callbackContainer != null)
            {
                _callbackContainer.Callback("Step One");
                _callbackContainer.Callback("Step Two");
                _callbackContainer.Callback("Step Three");
            }
        }

        void ICOMCallbackTestServer.Dispose()
        {
            if (_callbackContainer != null)
                _callbackContainer.Callback("Done");
        }
    }
}

This is the Delphi CallbackContainer:

type
  TCOMCallbackMethod = reference to procedure(AMessage: string);

  TCOMCallbackContainer = class(TAutoIntfObject, ICOMCallbackContainer)
  private
    FCallbackMethod: TCOMCallbackMethod;
    procedure Callback(const message: WideString); safecall;
  public
    constructor Create(ACallbackMethod: TCOMCallbackMethod);
    destructor Destroy; override;
  end;

//  ...

constructor TCOMCallbackContainer.Create(ACallbackMethod: TCOMCallbackMethod);
var
  typeLib: ITypeLib;
begin
  OleCheck(LoadRegTypeLib(LIBID_COMCallbackTestServer,
                          COMCallbackTestServerMajorVersion,
                          COMCallbackTestServerMinorVersion,
                          0,
                          {out} typeLib));
  inherited Create(typeLib, ICOMCallbackContainer);
  FCallbackMethod := ACallbackMethod;
end;

destructor TCOMCallbackContainer.Destroy;
begin
  FCallbackMethod := nil;

  inherited Destroy;
end;

procedure TCOMCallbackContainer.Callback(const message: WideString);
begin
  if Assigned(FCallbackMethod) then
    FCallbackMethod(message);
end;

TCOMCallbackContainer inherites from TAutoIntfObject so it implements IDispatch. I don't know if I'm doing the right thing in the constructor. I'm not as familiar with how to use IDispatch as I would like to be.

This is the Delphi COM Client:

procedure TfrmMain.FormCreate(Sender: TObject);
begin
  FServer := CoCOMCallbackTestServer_.Create as ICOMCallbackTestServer;

  //  Increments RefCount by 2, expected 1
  FServer.CallbackContainer := TCOMCallbackContainer.Create(Process_Callback);
end;

procedure TfrmMain.FormDestroy(Sender: TObject);
begin
  //  Decrements RefCount by 0, expected 1
  FServer.CallbackContainer := nil;

  FServer.Dispose;
  FServer := nil;
end;

procedure TfrmMain.btnBeginProcessClick(Sender: TObject);
begin
  FServer.RunCOMProcess;
end;

procedure TfrmMain.Process_Callback(AMessage: string);
begin
  mmoProcessMessages.Lines.Add(AMessage);
end;

The instance of TCOMCallbackContainer above never gets destroyed because the RefCount never gets below 2.

So my question is, why does assigning my callback container object to the COM property increase the reference count by two, and why does assigning nil to the COM property not decrease the reference count at all?

EDIT

I created TMyInterfacedObject (identical to TInterfacedObject) and used it as a base class for TCOMCallbackContainer. I put break-points in each method of TMyInterfacedObject. At each break-point I recorded the call-stack (and some other info). For each method that updates RefCount, the number at the end of the line shows the new value of RefCount. For QueryInterface, I included the IID and the corresponding interface name (found via Google) and the result of the call.

TfrmMain.FormCreate -> TCOMCallbackContainer.Create -> TInterfacedObject.NewInstance:  1
TfrmMain.FormCreate -> TCOMCallbackContainer.Create -> TInterfacedObject.AfterConstruction:  0
CLR -> TInterfacedObject.QueryInterface("00000000-0000-0000-C000-000000000046" {IUnknown}):  S_OK
CLR -> TInterfacedObject.QueryInterface -> TObject.GetInterface -> _AddRef:  1
CLR -> TInterfacedObject.QueryInterface("C3FCC19E-A970-11D2-8B5A-00A0C9B7C9C4" {IManagedObject}):  E_NOINTERFACE
CLR -> TInterfacedObject.QueryInterface("B196B283-BAB4-101A-B69C-00AA00341D07" {IProvideClassInfo}):  E_NOINTERFACE
CLR -> TInterfacedObject._AddRef:  2
CLR -> TInterfacedObject.QueryInterface("ECC8691B-C1DB-4DC0-855E-65F6C551AF49" {INoMarshal}):  E_NOINTERFACE
CLR -> TInterfacedObject.QueryInterface("94EA2B94-E9CC-49E0-C0FF-EE64CA8F5B90" {IAgileObject}):  E_NOINTERFACE
CLR -> TInterfacedObject.QueryInterface("00000003-0000-0000-C000-000000000046" {IMarshal}):  E_NOINTERFACE
CLR -> TInterfacedObject.QueryInterface("00000144-0000-0000-C000-000000000046" {IRpcOptions}):  E_NOINTERFACE
CLR -> TInterfacedObject._Release:  1
CLR -> TInterfacedObject.QueryInterface("2AB7E954-0AAF-4CFE-844C-756E50FE6360" {ICOMCallbackContainer}):  S_OK
CLR -> TInterfacedObject.QueryInterface -> TObject.GetInterface -> _AddRef:  2
CLR -> TInterfacedObject._AddRef:  3
CLR -> TInterfacedObject._Release:  2

All of the break-points listed happened in the FServer.CallbackContainer := TCOMCallbackContainer.Create(Process_Callback); statement within TfrmMain.Create. In the Destroy method, particularly at the FServer.CallbackContainer := nil; statement, none of the break-points were hit.

I thought, perhaps, that maybe the COM library was being unloaded before the destructor was called, so I copied the FServer.CallbackContainer := nil; line to the end of the constructor. It made no difference.

The interfaces passed to the calls to QueryInterface do not seem to be available in the Delphi environment, so I'm going to try inheriting some of them into ICOMCallbackContainer on the C# side to make them available (after researching what they're supposed to do and how they're supposed to work).

EDIT 2

I tried implementing INoMarshal and IAgileObject just to see what would happen. I tried those two because they are both marker interfaces and there was nothing to actually implement. It changed the process a little bit, but not in any way that helped. It seems that if the CLR finds INoMarshal then it doesn't look for IAgileObject or IMarshal, and if it doesn't find INoMarshal, but finds IAgileObject then it doesn't look for IMarshal. (Not that this seems to matter, or even make sense to me.)

After adding INoMarshal to TCOMCallbackContainer:

...
CLR -> TInterfacedObject._AddRef:  2
CLR -> TInterfacedObject.QueryInterface(INoMarshal):  S_OK
CLR -> TInterfacedObject.QueryInterface -> TObject.GetInterface -> _AddRef:  3
CLR -> TInterfacedObject._Release:  2
CLR -> TInterfacedObject.QueryInterface(IRpcOptions):  E_NOINTERFACE
CLR -> TInterfacedObject._Release:  1
...

After adding IAgileObject to TCOMCallbackContainer:

...
CLR -> TInterfacedObject._AddRef:  2
CLR -> TInterfacedObject.QueryInterface(INoMarshal):  E_NOINTERFACE
CLR -> TInterfacedObject.QueryInterface(IAgileObject):  S_OK
CLR -> TInterfacedObject.QueryInterface -> TObject.GetInterface -> _AddRef:  3
CLR -> TInterfacedObject._Release:  2
CLR -> TInterfacedObject.QueryInterface(IRpcOptions):  E_NOINTERFACE
CLR -> TInterfacedObject._Release:  1
...

Answer 1

In managed code external COM interfaces are wrapped into Runtime Callable Wrapper (RCW). Unlike a raw COM interface RCW lifespan is determined by the garbage collector that does not use reference counts. In your particular case it means that an assignment to null does not decrement the refCount right away.

COM object reference release can be forced by explicitly calling Marshal.ReleaseComObject:

     ICOMCallbackContainer ICOMCallbackTestServer.CallbackContainer
    {
        get { return _callbackContainer; }
        set { 

            if (_callbackContainer != null)
            {
                  Marshal.ReleaseComObject(_callbackContainer); // calls IUnknown.Release()
                  _callbackContainer = null;
            }

            _callbackContainer = value;
        }
    }