How can I optimize simple socket communication?

Question

Client sends a message. Server reads the message and writes a reply. Client reads the reply. Repeat. Each message is shorter than 500 bytes. Sockets are not closed.

I get around 800 request+responses/per second between two desktop PCs on LAN. Network activity on the hosts are barely noticeable.

If I don't do readReply (or do it in a separate thread), throughout explodes to like 30.000 msg/sec or more! This also peaks the network activity on the hosts.

My questions:

1. Is 800 msg/sec a reasonable number for a request/response protocol on a single socket?

2. How is it that removing the readReply call can increase performance like that???

3. What can be done to improve this, apart from using UDP? Any other protocol that might be used?

Server:

while (true) {
   message = readMessage();
   writeReply("Thanks");
}

Client:

while (true) {
   writeMessage("A message");
   reply = readReply();
}

Notes:

• I implemented this in both Java and Php and got about the same results.
• Ping latency is <1 ms

Answer 1

The basic problem is latency: the time it takes for a network frame/package to reach the destination.

For instance, 1 ms latency limits the speed to at most 1000 frames/second. Latency of 2 ms can handle 500 fps, 10 ms gives 100 fps etc..

1. In this case, managing 1600 fps (800*2) is expected when latency is 0.5 ms.

2. I think this is because you manage to send more data per frame. It will fill up the TCP buffer in the client after a while though.

3. Batch (pipeline) the messages if possible. Send 10 messages from the client in a batch and then wait for the server to reply. The server should send all 10 replies in a single chunk as well. This should make the speed 10x faster in theory.