race condition while working with file system

Question

I'm using a System.IO.FileSystemWatcher to get notified on file renaming inside a directory. This files are log files, created by a different process.

The event handler looks like this:

      private async void FileRenamedHandler(object sender, RenamedEventArgs e)
      {
          //when file is renamed
          //try to upload it to a storage
          //if upload is succesful delete it from disk 
      }

all looks good until now but i need to add a second method that iterates through the directory when this application starts in order to upload existing log files to storage

so

    public async Task UploadAllFilesInDirectory()
    {
        foreach (var file in Directory.GetFiles(_directoryPath))
        {
          await TryUploadLogAsync(file);
        }
    }

Problem is i get into race conditions like for example:

• file has just been renamed and FileRenamedHandler is triggered but the same fill would be also parsed by UploadAllFilesInDirectory method. In this moment i may upload the same file twice or i would get an exception when trying to delete it from disk because it has been already deleted.

  I can see more race condition cases with this code.

Any idea how i can solve this?

Thanks

Answer 1

You can use a ConcurrentDictionary to keep track of the items currently being processed, and let it worry about the thread safety.

Create the dictionary in which the key is the file path (or some other identifying object) and the value is...whatever. We're treating this as a set, not a dictionary, but there is no ConcurrentSet, so this will have to do.

Then for each file you have to process call TryAdd. If it returns true you added the object, and you can process the file. If it returns false then the file was there, and it's being processed elsewhere.

You can then remove the object when you're done processing it:

//store this somewhere
var dic = new ConcurrentDictionary<string, string>();

//to process each file
if (dic.TryAdd(path, path))
{
    //process the file at "path"
    dic.TryRemove(path, out path);
}

Answer 2

I would suggest to build a queue and store the files to be uploaded as some sort of job into the queue. If you process the items in the queue you can check the existence of every file before trying to upload it.