Why does FileSystemWatcher.Changed and StreamReader or FileStream lock the file even though I lock(...){...} it?

Question

I am writing some code to watch a (log) file. I use Notepad++ to update the file.
I listen to change events and sometimes the file is locked even though I have a lock statement around my file reading.
It is like StreamReader or FileStream releases the file but the operating system (win8 with dotnet4.5.1) keeps the lock for some more time?

I know of the caveat with the Changed event getting called twice but the lock statement should take care of it. I thought. Until now.

private static object _fileLock = new Object();
..
_watch.Changed += new FileSystemEventHandler(watch_Changed);
..
void watch_Changed(object sender, FileSystemEventArgs e)
{
    lock (_fileLock)
    {
        using (var sr = new FileStream(_pathAndFilename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
        {
            // read in file...
            sr.Close();
        }
    }
}

Answer 1

You are seriously confusing the idea of locking. Your lock statement in the Changed event handler is the kind of lock you'd use to prevent another thread from using a shared resource. We can't see such a resource being used in your snippet.

A lock on a file is an entirely different kind of animal. The FileShare enumeration is relevant, it says what kind of access other code has to the file. Including code in another process, something that's never the case with the lock keyword. You are very permissive in your own code, using FileShare.ReadWrite. Which allows any other process to both read and write to the file. Which is something you have to fret about a bit, does your code still work properly when another process can write to the file while you are reading it? In other words, can you deal with the file data changing while you are reading it. That's pretty doggone rare, you have to know a lot about what other processes are doing with this file. Also extremely hard to test.

That's enough introduction to the real problem. The FileSystemWatcher.Change event is fired while another process is writing the file. To be able to open that file in your event handler, the process that opened the file must have giving the sharing permission for the kind of access you are asking for. In your case, it must have at least specified FileShare.Read so that you have FileAccess.Read access.

Don't ever count on that always being the case. A pretty sane reasoning for a programmer that's writing a file is "there's no way that anybody else could ever read this file properly when I'm busy writing it". So he specifies FileShare.None. Kaboom in your code, no way you can open it.

Nothing you can do about it, it was that other programmer that said you wouldn't be able to make it work. With very high odds that he was completely correct about that.

You deal with this by reading the file later. After the process that is writing the file is done with it. Just remember the path to the file, put it on a thread-safe queue. Now you have a use for that lock statement. Use a timer to periodically try to open the file again.