GetQueuedCompletionStatus delayed

Question

I have written complex library for managing network communication based on iocp mechanism. Problem is that when server closes the connection by calling API method closesocket() this information is sometimes transmitted to client delayed by seconds or even minutes. My code for detecting connection closure looks like this (simplified):

ok = GetQueuedCompletionStatus(completion_port, &io_size, (PULONG_PTR)&context, &overlapped, 40);

if (!ok) {
   // something went broken
  DWORD err = GetLastError();
  if (err == ERROR_CONNECTION_REFUSED) {
         // connection failed
  } else if (err == ERROR_SEM_TIMEOUT) {
        // connection timeout
  } else if (err == ERROR_NETNAME_DELETED) {
        // connection closure - point of interest
  } else if (err != WAIT_TIMEOUT) {
        // unknown error
  }
} else {
    // process incomming or outgoing data
}

Why is this happening? I need to know about connection closure immediately to be able to connect to backup server (not so heavily loaded - disconnect is happening because of this).

Answer 1

I tried to experiment with linger parameter as Len wrote but this did not help. Adding call of shutdown() function just before closesocket() helped me. After analyzing packets reaching network interface on client (with WireShark) I have found that RST packet was replaced by FIN packet. Curiously that RST packet was not delayed. So operating system knew that connection was closed but by some unknown reason this information was transferred to application layer very delayed. I measured delays between 10 seconds and 4 minutes.

Answer 2

How are you closing the connection?

If you're just calling closesocket() then you are initiating a shutdown sequence which will attempt to ensure that all data that is currently pending will reach the destination. This can take time, especially if the network connection has been overloaded and datagrams have been lost and TCP retransmission is occurring.

If you want to close the connection straight away, and lose any pending data, then set linger to 0 and then close the socket. This will issue an RST on the connection and you'll get that much quicker.