How to tell when ReadFileEx() overlapped I/O has completed?

Question

HasOverlappedIoCompleted() doesn't work on asynchronous I/O begun with ReadFileEx() and WriteFileEx(). The code snippet at the bottom demonstrates this. In this example, ReadFileEx() reads from a pipe that has no input, so the read will not complete. But HasOverlappedIoCompleted() returns TRUE. If I change the call to an overlapped ReadFile() then HasOverlappedIoCompleted() returns FALSE as expected.

My question is: How can I find out whether an overlapped I/O request with callback has completed, without relying on the callback itself? In my application, the APC may have been queued but need not necessarily have run yet because the application may not yet have waited in an alertable state.

Thanks.

(Note GetOverlappedResult() doesn't help - it also returns TRUE.)

A bit more background: In the example I'm using ReadFileEx() because it is easy to demonstrate the problem. In my application I am calling WriteFileEx() repeatedly on a pipe instance. If the previous WriteFileEx() has not yet completed I must drop the message rather than send it (I must not have more than one pending write on the same pipe instance), but if the previous WriteFileEx() has completed then I must start the next one, even if the completion callback has not yet run.

Edit: A description of the problem scenario

1. The thread enters an alertable state (with one read APC queued).
2. The read APC begins: It queues a WriteFileEx() and sets a 'write pending' flag. It then queues a ReadFileEx().
3. The main thread begins work (non-alertable).
4. The queued read completes.
5. The queued write completes (after the read).
6. The main thread enters an alertable state.
7. The read APC is first in the queue so runs first: It looks at the 'write pending' flag and since it is still set it drops the write. In fact though the WriteFileEx() has completed, it just hasn't called its APC yet because the ReadFileEx() completed first.

Instead of testing my custom 'write pending' flag, I want to find out from the OS whether the WriteFileEx() has actually completed, even if the APC hasn't yet run.

---

#include <Windows.h>
#include <stdio.h>
#include <assert.h>

VOID CALLBACK readComplete(DWORD err, DWORD bytes, LPOVERLAPPED ovlp)
{
}

int main(int argc, char *argv[])
{
  HANDLE     hServer;
  OVERLAPPED serverOvlp = { 0 };
  HANDLE     hClient;
  DWORD      bytes;
  BYTE       buffer[16];
  BOOL       result;

  hServer = CreateNamedPipe("\\\\.\\pipe\\testpipe", PIPE_ACCESS_DUPLEX | FILE_FLAG_OVERLAPPED,
                            PIPE_TYPE_MESSAGE | PIPE_READMODE_MESSAGE | PIPE_WAIT, 
                            PIPE_UNLIMITED_INSTANCES, 0, 0, 5000, NULL);

  serverOvlp.hEvent = CreateEvent(NULL, FALSE, FALSE, NULL);

  ConnectNamedPipe(hServer, &serverOvlp);
  assert(GetLastError() == ERROR_IO_PENDING);

  hClient = CreateFile("\\\\.\\pipe\\testpipe", GENERIC_READ | GENERIC_WRITE,
                       0, NULL, OPEN_EXISTING, 0, NULL);

  GetOverlappedResult(hServer, &serverOvlp, &bytes, TRUE);

  /* Server starts an overlapped read */
//  result = ReadFile(hServer, buffer, sizeof(buffer), &bytes, &serverOvlp);
  result = ReadFileEx(hServer, buffer, sizeof(buffer), &serverOvlp, readComplete);

  if (HasOverlappedIoCompleted(&serverOvlp))
  {
    puts("Completed");
  }
  else
  {
    puts("Not completed");
  }


  return EXIT_SUCCESS;
}

Answer 1

The behaviour you are asking for seems wrong to me, because it involves a race condition. The issue only arises if you receive a message A while you are still sending a message B. At present, A is always ignored, i.e., no additional message is sent. The behaviour you are trying to get would result in an additional message being sent if and only if the server was busy processing work during the interval between A arriving and B completing. I think you should either always be ignoring A or always sending a response once B is completed.

However, you can get this behaviour if you're sure it is what you want. One solution would be to call QueueUserAPC from the ReadFileEx completion routine, to queue a call to a function which sends the reply. Since APCs are run in FIFO order, the new APC will definitely run after the APC that WriteFileEx has already queued.

Depending on the details, it might be cleaner for both completion routines to set flags, or add items to a queue, and have your main loop do the actual work.

If you need to poll for APCs (for example, because your main loop doesn't have any naturally occurring wait operations) you can use WaitForSingleObjectEx on a dummy event with a timeout of 0. It doesn't matter whether the event is signaled or not, all queued APCs will still be called.

Answer 2

I've changed your code a bit to achieve the CALLBACK. For this example to work, the pipe buffer length should not be 0. Also, I think that both ends of the pipe should have the OVERLAPPED flag set.

#include <Windows.h>
#include <stdio.h>
#include <assert.h>

#define BUFFSIZE 100
#define MYPIPE   "\\\\.\\pipe\\testpipe"

typedef struct {
  OVERLAPPED serverOvlp;        // Overlapped should always be first in structure
  CHAR       buffer[20];
} OVLP;


VOID CALLBACK readComplete(DWORD err, DWORD bytes, LPOVERLAPPED ovlp)
{
  OVLP *temp = (OVLP *) ovlp;
  printf("readComplete  err=%d  bytes=%d  buffer=%s\n", err, bytes, temp->buffer);
}

int main(void)
{
  HANDLE     hServer;
  HANDLE     hClient;

  OVLP       oServer;

  DWORD      bytes;
  CHAR       ClientBuffer[20] = "Test message";

  hServer = CreateNamedPipe(MYPIPE, PIPE_ACCESS_DUPLEX | FILE_FLAG_OVERLAPPED,
                            PIPE_TYPE_MESSAGE | PIPE_READMODE_MESSAGE | PIPE_WAIT,
                            PIPE_UNLIMITED_INSTANCES, BUFFSIZE, BUFFSIZE, 5000, NULL);

//-------------------------------------- CLIENT 
  hClient = CreateFile(MYPIPE, GENERIC_READ | GENERIC_WRITE,
                       0, NULL, OPEN_EXISTING, FILE_FLAG_OVERLAPPED, NULL);

  WriteFile(hClient,ClientBuffer,strlen(ClientBuffer)+1,&bytes,NULL);

//-------------------------------------- SERVER
  ConnectNamedPipe(hServer, &oServer.serverOvlp);
  if (HasOverlappedIoCompleted(&oServer.serverOvlp)) assert(GetLastError() != 0 );
  puts("Client Pipe connected\n");

  ReadFileEx(hServer, oServer.buffer, sizeof(oServer.buffer), (LPOVERLAPPED)&oServer, readComplete);

  SleepEx(INFINITE,TRUE);       // Creates an alertable event so CALLBACK is triggered

  if (HasOverlappedIoCompleted(&oServer.serverOvlp)) {
    puts("Completed");
  } else {
    puts("Not completed");
  }

  return EXIT_SUCCESS;
}