1) Excel registers itself as the default handler for .csv files. To open any file using the default file handler for the type, just do this:
Process.Start(@"c:\full\path\to\file.csv");
2) File.WriteAllText
should close the file at the end. The code you show above also has a line that reads File.Open(csvFileName...
that opens the file in exclusive write mode. Since you don't close that file, Excel can't open it until either a) the process terminates or b) the file is closed by the garbage collector. I bet the garbage collector hasn't yet run in cases where the access-denied error occurs.
Final recommendation: Remove the File.Open
and replace with Process.Start
and you may see different results.