Overcoming blocking writes in a server program

Question

I apologize ahead of the time for the lengthy wall of text.

I am writing a game server. Basically there is thread to accept connections, there are individual threads for each player, and a thread that processes all of the updating. The only responsibility of each individual player thread is to read and write packets to the actual client. Keep in mind that all of the I/O is done using the good ol' java.io package.

Anyways, when I finished writing some content, I decided that I wanted to perform some kind of a stress test. So I sent a bot flooder program to a friend of mine. The program rapidly connects fake players to the server. Rather than running it locally, I figured it would be ever so slightly more realistic if the connections were not local. What I discovered from the test was that the player update cycle times were pretty great, except that once about 200 players connected, the cycle times would begin spike up dramatically (we're talking from 15 milliseconds to sometimes over 9000). At first, I was thinking that it was somehow associated with the acceptor thread, because I noticed that when the cycles slowed down I was receiving "A player has connected from ..." messages less frequently. However, I found out that was not the cause after I decided to measure how long individual steps of an update cycle took. The bottleneck was in player updating. After I scrolled to the very bottom of the player update method, I instantly recognized the cause.

The problem is that at the end of each player update, the update packet is sent. The large, sometimes multiple kilobytes in size, packet is sent through the socket's output stream once it is constructed. That writing of the packet is a blocking operation, as many of you know. And that blocking is what was causing the seemingly random spikes in cycle times. Once a couple of hundred players would connect, the server would attempt to update all of those players. Sometimes though, one of the connections would be a bit slow. So the server would keep processing players until it attempted to process a player which was connected through a slow socket. Once it constructed the update packet, it would attempt to send it. However, the sending blocks for a long amount of time due to the slow socket. While the sending operation is blocking, none of the other players are being updated. As a result, cycles would sometimes take an extremely long time.

Now, I am kind of stuck on how to approach this problem. The ideal situation would be having the server construct the packet and hand off the packet to the individual player thread and have it send it (and merrily block in its own little thread). When the update packet would be constructed, the server thread would store the update packet and perhaps raise some flag that signals that a packet was constructed. This sounds fine, but the problem is that a lot of the time, the individual player thread is caught up in its own blocking operation. The player thread is blocking because it is waiting to receive a packet. So the update packet would be written once it finishes being blocked by the read, but I am afraid that the update packet would be sent way after it was actually constructed, which would make it seem that the player is lagging.

So here is when I ask you guys for suggestions. How would you approach this problem? Someone has recommended me to use NIO, but that would require me to rewrite a large portion of the code, so I would prefer to exhaust my possibilities because I take a different route.

Answer 1

As @Drakosha said in his comment, you can use a message queue with multiple threads servicing it.

Maybe something like this:

import java.util.concurrent.LinkedBlockingDeque;
import static java.lang.System.*;
class Message{
    String message;
    public Message(String message) {this.message=message;}
    public String toString(){return message;}
}
public class Test {
static LinkedBlockingDeque<Message> outgoing=new LinkedBlockingDeque<>();
    public static void main(String[] args) {
        for(int i=0;i<10;i++) new Player(outgoing,i).start();
        for(int i=0;i<3;i++)  new Sender(outgoing,i).start();
    }
}
class Player extends Thread{
    private LinkedBlockingDeque<Message> outgoing;
    private int id;
    public Player(LinkedBlockingDeque<Message> outgoing,int id){this.outgoing=outgoing; this.id=id;}
    public void run(){
        for(int i=0;i<10;i++){
            try {
                outgoing.putLast(new Message(String.format("Player %d's message",id)));
                Thread.sleep(20);
            } catch (InterruptedException e) {e.printStackTrace();}
        }
    }
}
class Sender extends Thread{
    private LinkedBlockingDeque<Message> outgoing;
    private int id;
    public Sender(LinkedBlockingDeque<Message> outgoing,int id){this.outgoing=outgoing; this.id=id;}
    public void run(){
        while(true){
            Message m = null;
            try {
                m = outgoing.takeFirst();
                //send message
                if(Math.random()>0.95){
                    out.printf("Sender %d is hanging! %n",id);
                    Thread.sleep(1000);//Slow socket is blocking!
                }
            } catch (InterruptedException e) {e.printStackTrace();}
            out.printf("Sender %d sent message \"%s\" %n",id,m);
        }
    }
}