Identifying Memory leak

Question

I guess I'm have a serious memory leak on one of my C# executables (it's a console application). The memory size keeps on increasing and overtime I need to restart the application to bring the memory usage down. I'm using a FileSystemWatcher and then when the file is available picking it up converting it to utf-8 and forwarding it. And then write to the console that the file is processed at this time. So it writes to the console every time it processes it.

I'm using Ants memory profiler and am pretty fresh at starting to use it. I can't figure it out with it. When I take a memory snapshot, it shows me:

namespace:System, Classname: byte[] --- This increases every time it processes the file and displays it on the console (by 40,000 bytes) and never goes back.

Is this correct.

Update:

class Class1
{
    private static FileSystemWatcher watcher = new FileSystemWatcher();

    public static void Main()
    {
        WatchFile();
        Console.ReadLine();
    }

    private static void WatchFile()
    {
        watcher.Path = @"N:\";
        watcher.NotifyFilter = NotifyFilters.LastWrite;
        watcher.Filter = "*.xml";
        watcher.Changed += new FileSystemEventHandler(convert);
        watcher.Error += new ErrorEventHandler(WatcherError);
        watcher.EnableRaisingEvents = true;

        Console.WriteLine("Press \'q\' to quit.");

        while (Console.Read() != 'q');
    }

    public static string CrL = "\r\n";

    private static object lockObject = new object();

    private static void convert(object source, FileSystemEventArgs f)
    {
       string FileName;
       FileName = f.FullPath;

       string destinationFile = @"F:\test.xml";

       Thread.Sleep(2000);
       if (!Monitor.TryEnter(lockObject))
       {
           return;
       }
       try
       {
           watcher.EnableRaisingEvents = false;
           Thread.Sleep(2000);

           var doc = new XmlDocument();
           doc.Load(FileName);

           XmlWriterSettings settings = new XmlWriterSettings { Encoding = Encoding.UTF8, Indent = true };

           using (var writer = XmlWriter.Create(destinationFile, settings))
           {
               doc.Save(writer);
           }

           Console.WriteLine("File Copied" + "  " + DateTime.Now.ToString("HH:mm:ss tt")); 
           Console.WriteLine("Press \'q\' to quit."); 
           Console.Write(CrL);
       }
       catch (Exception e)
       {
           Console.WriteLine("The process failed: {0}", e.ToString());
       }
       finally
       {
           Monitor.Exit(lockObject);
           watcher.EnableRaisingEvents = true;
           GC.Collect();
           GC.WaitForPendingFinalizers();
           GC.Collect();
       }
    }

    private class Utf8StringWriter : StringWriter
    {
        public override Encoding Encoding { get { return Encoding.UTF8; } }
    }

    private static void WatcherError(object source, ErrorEventArgs e)
    {
        Exception watchException = e.GetException();
        watcher = new FileSystemWatcher();
        while (!watcher.EnableRaisingEvents)
        {
            try
            {
                WatchFile();
                Console.WriteLine("I'm Back!!");
            }
            catch
            {
                Thread.Sleep(2000);
            }
        }
    }
}

Answer 1

Similar question here - Memory leak while using Threads

From what I understand from the solution, writing to Console and using Thread.Sleep was the issue.

Also, maybe applicable (or not) - http://support.microsoft.com/kb/2628838

Download hotfix (Reliability Update 2 for Framework 4.0) file from here- http://support.microsoft.com/kb/2600217

Answer 2

Try it like this:

var doc = new XmlDocument();
doc.Load(FileName);

XmlWriterSettings settings = new XmlWriterSettings { Encoding = Encoding.UTF8, Indent = true };

using (var writer = XmlWriter.Create(destinationFile, settings))
{
    doc.Save(writer);
}
doc = null;

If you don't set doc to null, the GC may see it still as an active reference and not free the memory. The GC has its own logic for garbage collection and forcing a full garbage collection is not the best practice (Best Practice for Forcing Garbage Collection in C#).

Also I don't believe that you have a memory leak; it appears that the GC hasn't been triggered to free the memory. By what do you mean when you say the memory keeps on increasing? What level does it reach when you feel you need to bring it back down?

EDIT

See this link for options on reading XML files: Deciding on when to use XmlDocument vs XmlReader

When working with very large XML files, it possible to get XMLDocuments allocated on the large object heap(LOH) which can cause fragmentation in the LOH and lead to OutOfMemory Exceptions.

XMLReader may be a better fit for you here then.