Using StreamWriter to write to different files in a Parallel.Foreach

Question

I am attempting to write separate collections to seperate csv in a Parallel.Foreach loop. When I run this, I get the following error:

The process cannot access the file 'C:\temp\OCRProcess\2011\49\OCRProcess-2011-49.csv' because it is being used by another process.

When I look at all the worker processes, they are opening different files, none of them are using the same file location. From what I can tell, each thread (worker process) is not conflicting.

public void GenerateCSVFile(List<CSVFile> csvFiles)
    {
        Parallel.ForEach(csvFiles, (csvfile, state, index) =>
            {
                var fileLocation = _storageLocation + csvfile.Type + s + csvfile.Year + s + csvfile.Day + s + csvfile.Type + "-" +
                           csvfile.Year + "-" + csvfile.Day + ".csv";

                if (!File.Exists(fileLocation))
                {
                    File.Create(fileLocation);
                }

                using (var streamWriter = new StreamWriter(fileLocation))
                {
                    foreach (var csvRecord in csvfile.CSVRecords)
                    {
                        streamWriter.WriteLine(csvRecord.Object_Id + "," + csvRecord.DocumentName + "," + csvRecord.BCI_DCN + "," + csvRecord.CreationDate);
                    }
                }                    
            });
    }

Here is the CSVFile and CSVRecord classes just incase.

public sealed class CSVRecord
{
    public String BCI_DCN { get; set; }
    public String Object_Id { get; set; }
    public String DocumentName { get; set; }
    public String CreationDate { get; set; }
}



public sealed  class CSVFile
{
    public List<CSVRecord> CSVRecords { get; set; }
    public String Year { get; set; }
    public String Type { get; set; }
    public String Day { get; set; }

    public CSVFile()
    {
        CSVRecords = new List<CSVRecord>();
    }
}

Answer 1

The issues is due to File.Create(fileLocation), which returns a FileStream and keeps the file open. When the StreamWriter attempted to open this, it was already open and caused the error.

To correct the problem, removed the following IF statement:

if (!File.Exists(fileLocation))
{
    File.Create(fileLocation);
}

And updated the USING statement as followed. By adding the TRUE parameter, it allows StreamWriter to append to the file if it exists, otherwise create it.

using (var streamWriter = new StreamWriter(fileLocation, true))
{
    foreach (var csvRecord in csvfile.CSVRecords)
    {
        streamWriter.WriteLine(csvRecord.Object_Id + "," + csvRecord.DocumentName + "," + csvRecord.BCI_DCN + "," + csvRecord.CreationDate);
    }
}