Indy10, 400ms for a simple TCP/IP request and response

Question

I can't figure out why a simple request and response is taking 400 ms to complete. It only needs under 1 ms to complete on localhost (loopback). When I make a request from my virtual machine to my main development machine it takes 400 ms to complete. It should take max 40 ms. This is how much it takes max for a HTTP request, so TCP should be faster. Here is the code for client and server. I just can't see where I loose time. I can profile if you need more info.

The code is Indy 9 and 10 compatible that is why the IFDEF-s. Also the connection is already established, it takes 400 ms without the connect part, only data send and response.

function TIMCClient.ExecuteConnectedRequest(const Request: IMessageData): IMessageData;
var
  DataLength: Int64;
  FullDataSize: Int64;
  IDAsBytes: TIdBytes;
  IDAsString: ustring;
begin
  Result := AcquireIMCData;
  FAnswerValid := False;

  with FTCPClient{$IFNDEF Indy9}.IOHandler{$ENDIF} do
  begin
    Request.Data.Storage.Seek(0, soFromBeginning);
    DataLength := Length(Request.ID) * SizeOf(uchar);
    FullDataSize := DataLength + Request.Data.Storage.Size + 2 * SizeOf(Int64);

    SetLength(IDAsBytes, DataLength);
    Move(Request.ID[1], IDAsBytes[0], DataLength);

    // write data
    {$IFDEF Indy9}WriteInteger{$ELSE}Write{$ENDIF}(FullDataSize);
    {$IFDEF Indy9}WriteInteger{$ELSE}Write{$ENDIF}(DataLength);
    {$IFDEF Indy9}WriteBuffer{$ELSE}Write{$ENDIF}(IDAsBytes{$IFDEF Indy9}[0]{$ENDIF}, DataLength);
    {$IFDEF Indy9}WriteInteger{$ELSE}Write{$ENDIF}(Request.Data.Storage.Size);
    {$IFDEF Indy9}WriteStream{$ELSE}Write{$ENDIF}(Request.Data.Storage);

    // set the read timeout
    ReadTimeout := FExecuteTimeout;
    FullDataSize := ReadInt(FTCPClient);

    // read the message ID
    SetLength(IDAsBytes, 0);
    DataLength := ReadInt(FTCPClient);
    ReadBuff(FTCPClient, DataLength, IDAsBytes);

    if DataLength > 0 then
    begin
      SetLength(IDAsString, DataLength div SizeOf(uchar));
      Move(IDAsBytes[0], IDAsString[1], DataLength);
      Result.ID := IDAsString;
    end;

    // read the message data
    DataLength := ReadInt(FTCPClient);
    ReadStream(Result.Data.Storage, DataLength, False);
    Result.Data.Storage.Seek(0, soFromBeginning);

    // we were succesfull
    FAnswerValid := True;
  end;
end;

The server side:

procedure TIMCServer.OnServerExecute(AContext: TIMCContext);
var
  Request: IMessageData;
  Response: IMessageData;
  DataLength: Int64;
  FullDataSize: Int64;
  IDAsBytes: TIdBytes;
  IDAsString: ustring;
begin
  with AContext.Connection{$IFNDEF Indy9}.IOHandler{$ENDIF} do
  begin
    ReadTimeout := FExecuteTimeout;
    //read the data length of the comming response
    FullDataSize := ReadInt(AContext.Connection);

    // Acquire the data objects
    Request := AcquireIMCData;
    Response := AcquireIMCData;

    // read the message ID
    DataLength := ReadInt(AContext.Connection);
    ReadBuff(AContext.Connection, DataLength, IDAsBytes);

    if DataLength > 0 then
    begin
      SetLength(IDAsString, DataLength div SizeOf(uchar));
      Move(IDAsBytes[0], IDAsString[1], DataLength);
      Request.ID := IDAsString;
    end;

    // read the message data
    DataLength := ReadInt(AContext.Connection);
    ReadStream(Request.Data.Storage, DataLength, False);

    Request.Data.Storage.Seek(0, soFromBeginning);
    try
      // execute the actual request handler
      FOnExecuteRequest(Request, Response);
    finally
      // write the data stream to TCP
      Response.Data.Storage.Seek(0, soFromBeginning);
      DataLength := Length(Response.ID) * SizeOf(uchar);
      FullDataSize := DataLength + Response.Data.Storage.Size + 2 * SizeOf(Int64);

      // write ID as binary data
      SetLength(IDAsBytes, DataLength);
      Move(Response.ID[1], IDAsBytes[0], DataLength);

      // write data
      {$IFDEF Indy9}WriteInteger{$ELSE}Write{$ENDIF}(FullDataSize);
      {$IFDEF Indy9}WriteInteger{$ELSE}Write{$ENDIF}(DataLength);
      {$IFDEF Indy9}WriteBuffer{$ELSE}Write{$ENDIF}(IDAsBytes{$IFDEF Indy9}[0]{$ENDIF}, DataLength);
      {$IFDEF Indy9}WriteInteger{$ELSE}Write{$ENDIF}(Response.Data.Storage.Size);
      {$IFDEF Indy9}WriteStream{$ELSE}Write{$ENDIF}(Response.Data.Storage);
    end;
  end;

The same slow communication was reported by one of the users of my code. He also tested from a virtual machine to a physical machine.

UPDATE:

The following code executes in 2-3 ms between same two machines. Its Indy10, smallest possible case.

procedure TForm2.Button1Click(Sender: TObject);
var
  MyVar: Int64;
begin
  TCPClient.Host := Edit1.Text;
  TCPClient.Port := StrToInt(Edit2.Text);
  TCPClient.Connect;
  try
    stopwatch := TStopWatch.StartNew;
    MyVar := 10;
    TCPClient.IOHandler.Write(MyVar);
    TCPClient.IOHandler.ReadInt64;
    stopwatch.Stop;
    Caption := IntToStr(stopwatch.ElapsedMilliseconds) + ' ms';
  finally
    TCPClient.Disconnect;
  end;
end;

procedure TForm2.TCPServerExecute(AContext: TIdContext);
var
  MyVar: Int64;
begin
  if AContext.Connection.IOHandler.InputBuffer.Size > 0 then
  begin
    MyVar := 10;
    AContext.Connection.IOHandler.ReadInt64;
    AContext.Connection.IOHandler.Write(MyVar);
  end;

end;

Answer 1

Solved the problem. It was easy when you find out what to do and where the problem is. Indy simply did not sent my data straight away. I had to add

OpenWriteBuffer
CloseWriteBuffer

calls to make Indy send the data when I want it. A lot of trouble over a simple misunderstanding of internal workings. Maybe this will spare someone some time.

When the buffer is closed, the data is sent immediately!