Guideline/best practices on what traits/categories to give to tests

Question

Is there any standard on what traits/categories to give to tests?

Here is an example:

using Xuint;

public class XUnitTest1
{
    [Fact]
    [Trait("Category", "CI")]
    public void TestMethodX1() { ... }

    [Fact]
    [Trait("Category", "CI")]
    public void TestMethodX2() { ... }

    [Fact]
    [Trait("Category", "CI")]
    [Trait("Runningtime", "Short")]
    [Trait("Priority", "2")]
    [Trait("Owner", "Terje")]
    public void TestMethodX3()
    {
        var sut = new SomeClasses.VerySimpleMath();
        int result = sut.Add(2, 3);
        Assert.Equal(result, 5)
    }
}

I guess that this is mostly used to reduce the time of executing when some are long but is there any standard? Should one bother starting giving traits/categories right from the start or only when tests are taking longer to run or when the need arises? What kind or traits/categories do people usually end up with?

Answer 1

Do not ask for "best practices". (Particularly not here, since people will become snarky about it.)

Software development is a mind game. The things we do, we do because it makes reasoning about our code easier for ourselves. If you mindlessly ape the behaviour of a successful practitioner without understanding his problems and why this behaviour is a solution for them, you will not gain his expertise.

This is very different from other careers. To become a successful pole vaulter, you must possess a lot of understanding and motivation, but you must also practice the movements again and again, tirelessly. In such a field, copying the behaviour of a champion is useful; it really does take a long time of practice to internalize the movements you need to make a successful vault, and the more often you repeat them, the better.

Creating software isn't like that. There is no sequence of movements that brings you success; there is no daily exercise that will hone your skill if you repeat it tirelessy without change. (I suspect that part of the success of the "design pattern" meme, and the more recent "Code Kata" meme, is due to the misguided hope among beginners that there is such a thing.)

Why, then do we have design patterns, code guidelines, or best practices? They are regularities that professionals who have solved certain problems have notices and considered them useful have described and named in the hope that others could profit from them as well. Sometimes this works. But it works well only if you actually do have the same problem that the author had. If you apply a solution that served someone else well to a different situation, it won't work well at all. In order to understand which useful solution to copy, you must understand the problem that the author had, what problem you have, and whether their nature is similar enough that re-applying the solution makes sense.

Back from the rant, on to your question: what problem do you hope to solve with traits on tests? (I freely confess that I've never used any, and had never even heard of thembeing used before you asked.)

Are your test suites too slow and you want to cherry-pick subsets of your tests to save time? I strongly advise against that; a test suite is supposed to be run completely, since you never know where or when unforeseed effects of a change will break some of your existing code. Do you want to impose an ordering on them? That's not a good idea; tests should be independent enough that they behave exactly the same no matter what order they're run in.

Did you just learn of the feature and now think "Hmm, I wonder what useful things I can do with that?" In that case you should store the knowledge away and calmly wait for a situation where traits can do something useful for you. If you do have a problem, i.e. your tests aren't doing what you need them to do, tell us a bit more about the problem, so that we understand it, and perhaps we can advise you.

Answer 2

You may want to use traits for

• Selectively running tests. For example, I use an Integration trait so that my dev machine test runner doesn't constantly try to run DB tests. You could also perhaps use a Nonessential trait to mark that if a test fails it won't stop the build from passing
• Metrics, so that you can (at a glance) see how many test are covering a particular module, how many of those are integration tests, etc