Skipping a whole test class in xUnit.net

Question

Is it possible to skip all tests from a specific class like in NUnit

[TestFixture]
[Ignore("Reason")]
public class TestClass {
}

Answer 1

No - there is no such facility at present,

One quick way of achieving the effect in xUnit is to comment out the public - private classes are not reflected over (obviously it won't appear on the skip list that way though).

Consider logging it as an Issue on CodePlex if you feel the need for this is sufficiently common (I personally can't imagine upvoting it as I simply don't run into cases where I need to skip an entire Test Class worth of tests).

UPDATE: Another way is to put a TraitAttribute on the class and then (assuming you're using the xunit.console runner) filter it out by running with /-trait traitName. (e.g. you can achieve ExplicitAttribute, some aspects of the BDD frameworky technique of Pending tests and similar semantics that way - of course the big problem is they don't show up in any reports when using any of these filtering techniques)

UPDATE 2: You can do

const string skip = "Class X disabled";

[Fact(Skip=skip)]
void Test() {}

Then you can change to to const string skip = null to undo the skip. The (dis)advantage of this is that the test is still shown as a Skipped test in the test list, generally with a reason included in the test run report (vs making it private which makes it likely to be forgotten)

Answer 2

Here is my hack to avoid error xUnit1000: Test classes must be public (checked on single Fact, I think Theories can be hacked this way, too).

// Uncomment to enable tests
//public class FactSwitch : FactAttribute { } // public! ahh, a bug!

// Uncomment to disable tests
internal class FactSwitch : Attribute { }

public class MyTests
{
    [FactSwitch]
    public void MyTest1()
    {
        "it".ShouldBe("it");
    }
}

(3 years later)

While searching for the same solution I found there are better ways to do the same. Let's rewrite the example above in a way Ruben Bartelink suggested (continuation of his idea).

public class MyTests
{
    //const string SkipOrNot = null; // Run all tests
    const string SkipOrNot = "reason"; // Skip all tests

    [Fact(Skip = SkipOrNot)]
    public void MyTest1()
    {
        "it".ShouldBe("it");
    }
}

Nathan Cooper suggested a good improvement for my idea:

public class MyTests
{
    // Uncomment to disable tests
    //private class FactAttribute : Attribute { }

    [Fact]
    public void MyTest1()
    {
        "it".ShouldBe("it");
    }
}

So I like both ideas from Ruben and Nathan. There is a subtle difference between using Skip="something" (Ruben) and not using Skip at all. Using "Skip" will put all your tests in a "Skipped tests" warning zone, while "FactAttribute : Attribute" will hide them.

Answer 3

Here's another hack that requires minimal changes to code

using FactAttribute = System.Runtime.CompilerServices.CompilerGeneratedAttribute;
using TheoryAttribute = System.Runtime.CompilerServices.CompilerGeneratedAttribute;

Any compatible attribute can be used for the replacement.

If you also use the InlineDataAttribute then you'll need to define a replacement as I don't think there's an existing compatible attribute.

using InlineDataAttribute = DummyDataAttribute;

[AttributeUsage(AttributeTargets.Method, AllowMultiple = true)]
internal class DummyDataAttribute : Attribute
{
    public DummyDataAttribute(params object[] data)
    {
    }
}

Answer 4

I've found yet another way of temporary disabling entire class without compiler warning.

Disabled:

[System.Diagnostics.CodeAnalysis.SuppressMessage("Usage", "xUnit1000:Test classes must be public", Justification = "Disabled")]//*/
/*
public /**/class DatabaseTests
{
}

to enable move the /* one line up (i.e. using alt+up):

/*
[System.Diagnostics.CodeAnalysis.SuppressMessage("Usage", "xUnit1000:Test classes must be public", Justification = "Disabled")]//*/
public /**/class DatabaseTests
{
}

Note that using full namespace path for SupressMessage does not mess up with your usings.

Answer 5

You need to set the your class access level as as internal and surpress message as @Miq did:

[System.Diagnostics.CodeAnalysis.SuppressMessage("Usage", "xUnit1000:Test classes must be public", Justification = "Disabled")]
internal class MyClassThatIsNotATestClass
{ ... }

Answer 6

You can create LocalOnlyFactAttribute

    public class LocalOnlyFactAttribute : FactAttribute
    {
        //uncomment to run on local
        //const string skip = null; 

        //keep this to avoid slow running tests on other env
        const string skip = "Disabled slow running tests."; 

        public override string Skip { get => skip; set => base.Skip = value; }
    }   

Answer 7

As far as I know, the simplest way to dynamically skip a whole xUnit test class at runtime is to use the TestFrameworkAttribute at the assembly level, to point to a class that implements the ITestFramework interface (or inherits from XunitTestFramework, which is simpler) and which overrides the CreateDiscoverer() method to return another class, that implements the ITestFrameworkDiscoverer interface (or inherits from XunitTestFrameworkDiscoverer, which is simpler), where you can finally override the IsValidTestClass() method, to decide whether a class should be skipped or not.

Here is some sample code:

[assembly: TestFramework("MyNamespace.Xunit.MyTestFramework", "MyAssembly")]

namespace MyNamespace.Xunit
{
    public class MyTestFramework : XunitTestFramework
    {
        public MyTestFramework(IMessageSink messageSink)
            : base(messageSink)
        {
        }

        protected override ITestFrameworkDiscoverer CreateDiscoverer(
            IAssemblyInfo assemblyInfo)
            => new MyTestFrameworkDiscoverer(
                assemblyInfo,
                SourceInformationProvider,
                DiagnosticMessageSink);
    }

    public class MyTestFrameworkDiscoverer : XunitTestFrameworkDiscoverer
    {
        public MyTestFrameworkDiscoverer(
            IAssemblyInfo assemblyInfo,
            ISourceInformationProvider sourceProvider,
            IMessageSink diagnosticMessageSink,
            IXunitTestCollectionFactory collectionFactory = null)
            : base(
                assemblyInfo,
                sourceProvider,
                diagnosticMessageSink,
                collectionFactory)
        {
        }

        protected override bool IsValidTestClass(ITypeInfo type)
            => base.IsValidTestClass(type) &&
               FilterType(type);

        protected virtual bool FilterType(ITypeInfo type)
        {
            // Insert your custom filter conditions here.
            return true;
        }
    }
}

Tested with xUnit 2.4.1.

We are using it in Pomelo.EntityFrameworkCore.MySql (see AssemblyInfo.cs and MySqlXunitTestFrameworkDiscoverer.cs) (a bit more complex than the sample code here).

Answer 8

Adding a reason almost after one year after the initial question. I have a set of tests which are calling real server apis, and I would like to run then on demand. With nUnit, it has Ignore attribute : with that set, test runner will skip those tests, but I can still manually run it.

xUnit has no such feature. The nearest one is setting such a class level attribute, and comment it out when I want to run it.

Answer 9

You could achieve this through a custom ITestClassCommand.

See http://mariangemarcano.blogspot.be/2010/12/xunitnet-running-tests-testcategory.html

Answer 10

Consider creating LocalOnlyFactAttribute, which can be reused across multiple test files.

    public class LocalOnlyFactAttribute : FactAttribute
    {
        //uncomment to run on local
        //const string skip = null; 

        //keep this to avoid slow running tests on other env
        const string skip = "Disabled slow running tests."; 

        public override string Skip { get => skip; set => this.Skip = value; }
    }