Using StreamReader / StreamWriter to grab logs causes program to cease responding

Question

I'm attempting to use StreamReader and StreamWriter to grab a temporary output log (.txt format) from another application.
The output log is always open and constantly written to.
Unhelpfully if the application closes or crashes, the log file ends up deleted - hence the need for a tool that can grab the information from this log and save it.

What my program currently does is:

• Create a new .txt file, and stores the path of that file as the string "destinationFile".
• Finds the .txt log file to read, and stores the path of that file as the string "sourceFile"
• It then passes those two strings to the method below.

Essentially I'm trying to read the sourceFile one line at a time.
Each time one line is read, it is appended to destinationFile.
This keeps looping until the sourceFile no longer exists (i.e. the application has closed or crashed and deleted its log).

In addition, the sourceFile can get quite big (sometimes 100Mb+), and this program may be handling more than one log at a time.
Reading the whole log rather than line by line will most likely start consuming a fair bit of memory.

private void logCopier(string sourceFile, string destinationFile)
{
    while (File.Exists(sourceFile))
    {
        string textLine;
        using (var readerStream = File.Open(sourceFile, 
                                            FileMode.Open, 
                                            FileAccess.Read, 
                                            FileShare.ReadWrite))
        using (var reader = new StreamReader(readerStream))                                      
        {
            while ((textLine = reader.ReadLine()) != null)
            {
                using (FileStream writerStream = new FileStream(destinationFile, 
                                                                FileMode.Append,
                                                                FileAccess.Write))
                using (StreamWriter writer = new StreamWriter(writerStream))
                {
                    writer.WriteLine(textLine);
                }                   
            } 
        }
    }
}

The problem is that my WPF application locks up and ceases to respond when it reaches this code.
To track down where, I put a MessageBox just before the writerStream line of the code to output what the reader was picking up.
It was certainly reading the log file just fine, but there appears to be a problem with writing it to the file.
As soon as it reaches the using (FileStream writerStream = new FileStream part of the code, it stops responding.

Is using the StreamWriter in this manner not valid, or have I just gone and dome something silly in the code?
Am also open to a better solution than what I'm trying to do here.

Answer 1

I've just solved the problem, and the issue was indeed something silly!

When creating the text file for the StreamWriter, I had forgotten to use .Dispose();. I had File.Create(filename); instead of File.Create(filename).Dispose(); This meant the text file was already open, and the StreamWriter was attempting to write to a file that was locked / in use.

The UI still locks up (as expected), as I've yet to implement this on a new thread as SteenT mentioned. However the program no longer crashes and the code correctly reads the log and outputs to a text file.

Also after a bit of refinement, my log reader/writer code now looks like this:

    private void logCopier(string sourceFile, string destinationFile)
    {
        int num = 1;
        string textLine = String.Empty;
        long offset = 0L;

        while (num == 1)
        {
            if (File.Exists(sourceFile))
            {

                FileStream stream = new FileStream(sourceFile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
                using (new StreamReader(stream))
                {
                    stream.Seek(offset, SeekOrigin.Begin);
                    TextReader reader2 = new StreamReader(stream);
                    while ((textLine = reader2.ReadLine()) != null)
                    {
                        Thread.Sleep(1);
                        StreamWriter writer = new StreamWriter(destinationFile, true);
                        writer.WriteLine(textLine);
                        writer.Flush();
                        writer.Close();
                        offset = stream.Position;
                    }
                    continue;
                }
            }
            else
            {
                num = 0;

            }
        }
    }

Just putting this code up here in case anyone else is looking for something like this. :)

Answer 2

Simply what I understand is you need to copy a file from source to destination which may be deleted at any time.

I'll suggest you to use FileSystemWatcher to watch for source file changed event, then just simply copy the whole file from source to destination using File.Copy.