ShowMessage in Delphi XE3 DllMain

Question

DLL below is compiled with XE3.

library MyDLL;

uses System.SysUtils, System.Classes, Vcl.Dialogs;

{$R *.res}

var
  II: Integer;

function Test: Integer;
begin
  Result := II;
end;

exports Test;

begin
  II := 5;
  ShowMessage('DLL prolog');
end.

When I call the function Test() from Delphi XE3 or Delphi 2007 program I get the correct result (5) in both cases. But the message "DLL prolog" is displayed only with Delphi 2007 program, not with Delphi XE3 program. Why?

Edit

And when I "run" dll from XE3 IDE with Delphi 2007 host ShowMessage() also does not work.

Answer 1

The key is in this function:

function MessageDlgPosHelp(const Msg: string; DlgType: TMsgDlgType;
  Buttons: TMsgDlgButtons; HelpCtx: Longint; X, Y: Integer;
  const HelpFileName: string): Integer;
begin
  if TOSVersion.Check(6) and UseLatestCommonDialogs and
     StyleServices.Enabled and StyleServices.IsSystemStyle then
    Result := DoTaskMessageDlgPosHelp('', Msg, DlgType, Buttons,
      HelpCtx, X, Y, HelpFileName)
  else
    Result := DoMessageDlgPosHelp(CreateMessageDialog(Msg, DlgType, Buttons),
      HelpCtx, X, Y, HelpFileName);
end;

In some cases, depending on whether or not the host application has a comctl32 v6 manifest, different branches of the if statement are chosen.

If the DoTaskMessageDlgPosHelp branch is chosen, then the ensuing call to TaskDialogIndirect fails with HRESULT code $80070057. This is a Win32 error code, ERROR_INVALID_PARAMETER.

If the DoMessageDlgPosHelp branch is chosen, then the dialog shows.

I'm not sure why TaskDialogIndirect is failing when called from a library initialization block, but I'm not entirely surprised. You are severely restricted in what can be done from DllMain and you should not be attempting to show dialogs from there.