Reading file right after it has been written I get all zeros (.net)

Question

I have a program that needs to load files from a directory as soon as they are written. I got the FileSystemWatcher to notify me of changes to the directory. Rather than check the event for what changed I just list the files and start processing all that I found.

To prevent trying to read a file which is still being written I have code like:

try {
    fs = fi.Open(FileMode.Open, FileAccess.ReadWrite,
                    FileShare.None);
    message = new byte[fs.Length];
    int br = fs.Read(message, 0, (int)fi.Length);
}
catch (Exception e) {
    // I'll get it next time around
    return;
}
finally {
    if (fs != null)
        fs.Close();
}

The problem is that for some files, about 1 in 200, the program read all zeros. File length is correct but the content appear to be all zero bytes. When I check the file latter I find it does contain actual correct data. I thought the way I was opening the file would prevent premature access to the file.

I'm testing this by copying files to the directory with the DOS command 'copy InFile_0* dropdir' (About 100 files per execution.) Possibly this command does the copy in two steps: 1) allocate space and 2) fill space and my program occasionally jumps in the middle of the two.

Any ideas on how to code this to be reliable?

Update: I don't have control over the writing program - it could be anything. Looks like I have to code defensively.

Answer 1

You're hitting a race condition. It's only going to get worse from here (with networked file systems etc) unless you fix it definitively.

Try having the program writing the files write each one using a "whatever.tmp" name, then close it, then rename it. When reading, ignore .tmp files.

Or, keep a zero-length file named "sentinel" or some such in the directory. Get the program writing the files to rewrite the sentinel file AFTER each successful write of another file. Then, don't attempt to read files whose modification date/times are >= the modification date/time of the sentinel file.

Or, if you have no control over the writer of the files, check each file's modification date/time against the current system date/time. Let the files age an appropriate amount (a few seconds if they're small, longer if they're larger) before attempting to read them.

Good luck. This is a notorious pain in the neck.

Answer 2

Well, I disagree with previous post by @Ollie Jones.

You already established exclusive access to the file, so no race condition problem.

I think you should examine the writer's behavior more carefully. And try reduce the interference on the file access with read only, share all access:

fi.Open(FileMode.Open, FileAccess.Read, FileShare.ReadWrite);

This could fail your read but will reduce write error. To decide when to read safely, you could check on file time or file size or whatever. If many files will be written subsequently you could start read the first file after second file created.