Arbitrary unit tests fail in VS2012 after switching from Moles to Fakes

Question

I have about 300 unit tests for an assembly which is part of a solution I originally started under VS2010. Numerous tests used the Moles framework provided by Micrsoft, but after upgrading to VS2012 (Update 2) I wanted to change the tests to use the officially supplied Fakes framework.

I updated the corresponding tests accordingly, which usually only involved creating a ShimsContext and some minor changes to the code:

Before

[TestMethod]
[HostType( "Moles" )]
public void MyUnitTest_CalledWithXyz_ThrowsException()
{
    // Arrange
    ...
    MGroupPrincipal.FindByIdentityPrincipalContextIdentityTypeString = 
        ( t1, t2, t3 ) => null;
    ...

    try
    {
        // Act
        ...
    }
    catch( Exception ex )
    {
        // Assert
        ...
    }       
}

After

[TestMethod]
public void MyUnitTest_CalledWithXyz_ThrowsException()
{
    using( ShimsContext.Create() )
    {
        // Arrange
        ...
        ShimGroupPrincipal.FindByIdentityPrincipalContextIdentityTypeString = 
            ( t1, t2, t3 ) => null;

        try
        {
            // Act
            ...
        }
        catch( Exception ex )
        {
            // Assert
            ...
        }       
    }
}

I've got different test classes in my test project, and when I run the tests I get arbitrary erros which I cannot explain, e.g.:

• Run tests for one class in Release mode => 21 tests fail / 15 pass
• Run tests for same class in Debug mode => 2 tests fail / 34 pass
• Run tests for same class again in Release mode => 2 tests fail / 34 pass
• Run all tests in the project => 21 tests fail / 15 pass (for the class mentioned above)

Same behaviour for a colleague on his system. The error messages are always TypeLoadExceptions such as

Test method ... threw exception: System.TypeLoadException: Could not load type 'System.DirectoryServices.Fakes.ShimDirectorySearcher' in the assembly 'System.DirectoryServices.4.0.0.0.Fakes,Version=4.0.0.0, Culture=neutral, PublicKeyToken=..."

In VS2012 itself the source code editor doesn't show any errors, Intellisense works as expected, mouse tooltips over e.g. ShimDirectorySearcher show where it is located etc. Furthermore, when I open the Fakes assembly that's being generated (e.g. System.DirectoryServices.4.0.0.0.Fakes.dll) with .NET Reflector, the type shown in the error message exists.

All the tests worked fine (in Debug and Release mode as well) before we switched from VS2010 to VS2012, but now we don't have a clue what's wrong here. Why does the result change in the ways described above? Why do we get TypeLoadExceptions even though the types do exist?

Unfortunately there is hardly any help available from Micrsoft or on the internet.

Answer 1

I don't quite understand why having the old .testsettings file from VS2010 is such a problem, but deleting it and adding a .runsettings file as suggested by MSDN did the job for me.

All problems were solved:

• All unit tests run (again) without problems
• Arbitrary combinations of tests run (again) without problems
• I can debug tests using Fakes (before I used to get test instrumentalisation errors)

Hope this helps others who run into problems, there doesn't seem to be too much information about Fakes yet.

One more thing regarding Code Coverage: This works (without having to configure any test settings) via menu Test => Analyze Code Coverage. For TFS build definitions you can enable code coverage for the build by choosing Process => Basic => Automated Tests => 1. Test Source. Now click into the corresponding text field and then on the ... button that is (only) shown when you click into the text field. Now choose Visual Studio Test Runner in the Test runner ComboBox. Now you can also choose Enable Code Coverage from the options.