DataSnap XE2 and TStream method parameters

Question

I am working on DataSnap project in Delphi XE2 using TCP/IP protocol that needs to pass a stream of binary data to the server as a method parameter. The problem I am running into is that there seems to be a size limit of about 32 KB on the stream contents. Beyond this limit the stream received at the server is empty. If I pass additional method parameters they arrive intact so it seems to be an issue at the parameter level.

Here is how the DataSnap service class is declared:

  TDataSnapTestClient = class(TDSAdminClient)
  private
    FSendDataCommand: TDBXCommand;
  public
    constructor Create(ADBXConnection: TDBXConnection); overload;
    constructor Create(ADBXConnection: TDBXConnection; AInstanceOwner: Boolean); overload;
    destructor Destroy; override;
    procedure SendData(Data: TStream);
  end;

The approach I am using should work, at least according to the article by Jim Tierney. That said, there apparently have been changes since Delphi 2009 that have broken Jim Tierney's sample code.

DataSnap Server Method Stream Parameters

Any ideas on how to resolve this issue would be greatly appreciated.

Answer 1

DataSnap transfers the data in 32k chunks. The receiving end has no way of knowing how many bytes will be received until after all chunks have been reassembled. Once all the data has been received, DataSnap doesn't set the size of the TStream that received the data, so you can't use it until you move it to another stream that knows how many bytes are in the stream.

I know that pulling 32k+ from a DataSnap server is not the same as pushing 32k+ to a DataSnap server, but this may work for you as well. Try running the TStream through this code after the DataSnap server finishes receiving the data:

procedure CopyStreamToMemoryStream(const ASource: TStream; var ADest: TMemoryStream; const Rewind: Boolean = True);
const
  LBufSize = $F000;
var
  LBuffer: PByte;
  LReadCount: Integer;
begin
  GetMem(LBuffer, LBufSize);
  ADest.Clear;
  try
    repeat
      LReadCount := ASource.Read(LBuffer^, LBufSize);
      if LReadCount > 0 then
        ADest.WriteBuffer(LBuffer^, LReadCount);
    until LReadCount < LBufSize;
  finally
    FreeMem(LBuffer, LBufSize);
  end;
  if Rewind then
    ADest.Seek(0, TSeekOrigin.soBeginning);
end;

I can't remember where I found this code (years ago), so I can't give credit where credit is due, but it has been working for me reliably for years now.

Answer 2

I got thinking about it and it occurred to me that transferring the data to another memory stream just wastes memory, especially if the file is very large. All we need to do is count the bytes and set the stream size, right?!

procedure FixStream(const AStream: TStream);
const
  LBufSize = $F000;
var
  LBuffer: PByte;
  LReadCount, StreamSize: Integer;
begin
  GetMem(LBuffer, LBufSize);
  try
    StreamSize := 0;
    repeat
      LReadCount := AStream.Read(LBuffer^, LBufSize);
      Inc(StreamSize, LReadCount);
    until LReadCount < LBufSize;
    AStream.Size := StreamSize;
  finally
    FreeMem(LBuffer, LBufSize);
  end;
end;

Do you want to give that a try? I'm not able to test the code right now or I would...