Indy 10 IdTCPSever Readbytes scrambling data

Question

I am trying to use Indy10 ReadBytes() in Delphi 2007 to read a large download of a series of data segments formatted as [#bytes]\r\n where #bytes indicates the number of bytes. My algorithm is:

1. Use ReadBytes() to get the [#]\r\n text, which is normally 10 bytes.
2. Use ReadBytes() to get the specified # data bytes.
3. Go to step 1 if more data segments need to be processed, i.e., # is negative.

This works well but frequently I don't get the expected text at step 1. Here's a short example after 330 successful data segments:

Data received from last step 2 ReadBytes(). NOTE embedded Step 1 [-08019]\r\n text.

Line|A033164|B033164|C033164|D033164|E033164|F033164|G033164|H033164|EndL\r|Begin 
Line|A033165|B033165|C033165|D033165|E033165|F033165|G033165|H033165|EndL\r|Begin 
Line|A033166|B033166|C033166|D033166|E033166|F033166|G033166|H033166|EndL\r[-08019]

\r\n|Begin Line|A033167|B033167|C033167|D033167|E033167|F033167|G033167|H033167|EndL\r|Begin Line|A033168|B033168|C033168|D033168|E033168|F033168|G033168|H033168|EndL\r|Begin

Socket data captured by WireShark.

0090 30 33 33 31 36 36 7c 42 30 33 33 31 36 36 7c 43 033166|B033166|C
00a0 30 33 33 31 36 36 7c 44 30 33 33 31 36 36 7c 45 033166|D033166|E
00b0 30 33 33 31 36 36 7c 46 30 33 33 31 36 36 7c 47 033166|F033166|G
00c0 30 33 33 31 36 36 7c 48 30 33 33 31 36 36 7c 45 033166|H033166|E
00d0 6e 64 4c 0d ndL.

No. Time Source Destination Protocol Length Info

2837 4.386336000 000.00.247.121 000.00.172.17 TCP 1514 40887 > 57006 [ACK] Seq=2689776 Ack=93 Win=1460 Len=1460
Frame 2837: 1514 bytes on wire (12112 bits), 1514 bytes captured (12112 bits) on interface 0
Ethernet II, Src: Cisco_60:4d:bf (e4:d3:f1:60:4d:bf), Dst: Dell_2a:78:29 (f0:4d:a2:2a:78:29)
Internet Protocol Version 4, Src: 000.00.247.121 (000.00.247.121), Dst: 000.00.172.17 (000.00.172.17)
Transmission Control Protocol, Src Port: 40887 (40887), Dst Port: 57006 (57006), Seq: 2689776, Ack: 93, Len: 1460
Data (1460 bytes)

0000 5b 2d 30 38 30 31 39 5d 0d 0a 7c 42 65 67 69 6e [-08019]..|Begin
0010 20 4c 69 6e 65 7c 41 30 33 33 31 36 37 7c 42 30 Line|A033167|B0
0020 33 33 31 36 37 7c 43 30 33 33 31 36 37 7c 44 30 33167|C033167|D0
0030 33 33 31 36 37 7c 45 30 33 33 31 36 37 7c 46 30 33167|E033167|F0

Does anyone know why this happens? Thanks

More information. We do socket reading from a single thread and don't call Connected() while reading. Here's relevant code snippet:

AClientDebugSocketContext.Connection.Socket.ReadBytes(inBuffer,byteCount,True);
numBytes := Length(inBuffer);
Logger.WriteToLogFile(BytesToString: '+BytesToString(inBuffer,0,numBytes),0);
Move(inBuffer[0], Pointer(Integer(Buffer))^, numBytes);

Answer 1

Embedded data like you describe, especially at random times, usually happens when you read from the same socket in multiple threads at the same time without adequate synchronization between them. One thread may receive a portion of the incoming data, and another thread may receive another portion of the data, and they end up storing their data in the InputBuffer in the wrong order. Hard to say for sure if that your problem since you did not provide any code. The best option is to make sure you never read from the same socket in multiple threads at all. That includes any calls to Connected(), since it performs a read operation internally. You should do all of your reading within a single thread. If that is not an option, then at least wrap your socket I/O with some kind of inter-thread lock, such as a critical section or mutex.

Update: You are accessing a TIdContext object via your own AClientDebugSocketContext variable. Where is that code being used exactly? If it is not in the context of the server's OnConnect, OnDisconnect, OnExecute, or OnException events, then you are reading from the same socket across multiple threads, because TIdTCPServer internally calls Connected() (which does a read) in between calls to the OnExecute event for that TIdContext object.