MSTest & AppDomains

Question

In some my project I notice that during executing unit tests under VSTS2008 its VSTestHost's memory consuming grows. As I have very many tests in my solution it leads to OutOfMemroyException eventually. That looks very strange for me as I was sure that MSTest creates a new AppDomain for each unit test. Otherwise how would it reset static fields? But if AppDomain is being created for each test than memory shouldn't leak. But it does.

So the question is: Should VS create AppDomain for each test class or not? If yes than how can I check that it does it. I tried tracing through ProcessExpolorer and Performance snap-in. A value of "Total appdomain unloaded" is always 0 during test run.

Answer 1

I don't think the unit test engine creates a new AppDomain for each test. Since creating an AppDomain is a relatively expensive operation, doing so for each test would slow down execution of unit tests considerably!

Visual Studio 2008 uses a seperate executable called vstesthost.exe to run unit tests. VS communicates with vstesthost.exe (how it does this I don't know) to tell it what tests to run. vstesthost.exe returns the execution results to VS which displays those results.

If you are getting OutOfMemoryExceptions when running your unit tests I would say that's a strong indicator that your code under test is actually not cleaning things up. Are you sure that you aren't retaining handles to unmanaged objects/memory? I would recommend running your unit tests under a Performance Analysis (you can do that by finding the unit test under the "Test View", right-clicking on it, and selecting "Create Performance Session"). This might shed some light at least on your object allocations.

Answer 2

MsTest creates one-app domain per Test assembly, unless you are using noisolation, in which case there is no AppDomain Isolation.

If you are seeing leaks, its probably a but in either your test code, or your product code. Make sure you aren't stuffing things into dictionaries and leaving them there.

Answer 3

I was wrong about having separate AppDomains for each unittest.

Here's evidence: a singleton

public class Singleton
{
    public static Singleton Instance = new Singleton();

    private Guid _token;
    private Singleton()
    {
        _token = Guid.NewGuid();
    }

    public Guid Token
    {
        get { return _token; }
    }
}

and two tests:

[TestClass]
public class UnitTest2
{
    [TestMethod]
    public void TestMethod1()
    {
        Console.WriteLine(Singleton.Instance.Token);
    }
}
[TestClass]
public class UnitTest1
{
    [TestMethod]
    public void TestMethod1()
    {
        Console.WriteLine(Singleton.Instance.Token);
    }
}

During executing both tests output the same guid.

Answer 4

Seen the same problem with large test runs. My theory is the following. Memory exhaustion in this case is due to the fact that MSTest test result files are XML. Therefore it needs to keep all the log results in memory until the end of the test run before serializing to disk. Hurray for XML :-)

I have posted this problem as a connect issue a while back and it should have been fixed in MSTest 10 (going 64 bit) but I haven't been able to verify this yet because of all the other problems we have moving to VS2010 and .NET 4.0.

Answer 5

The only way to dispose of a singleton is to dispose the appDomain. A singleton is a static holding onto itself, so it's basically a circular reference. True singletons do not get disposed until the appdomain goes away.

Answer 6

This does not seem to be solved in MSTest 2010. I am experiencing a lot of similar issues like this. Why does garbage collection not work in unit test?

My understanding was that the UT framework took care of disposing of all executed tests, but this does not seem to be the case with some singleton patterns that we have in code.