How do I create an environment where fixing tests is seen as a priority?

Question

I am a software engineer at a medium sized company. We have a fairly robust testing platform running on TeamCity. It does unit tests on every checkin, and a daily unit test/BVT run.

The problem is - we have a great deal of broken unit tests.

Quite often, I bring up the pointlessness of unit tests if they are constantly breaking and unmaintained. Being unable to see if a change has caused a regression removes most of the value of a unit testing platform.

I would like to get a seed planted that will create a culture of good habits - fixing tests when they're broken, seeing them as valuable, prioritizing the fixing of tests along with other work.

I've tried bribery (baked goods!), just plain asking, and speaking to team leads. Everyone says that it's a good idea, but I see to be the only one doing anything about it.

What is the best way to get started on encouraging others to fix their tests, and prioritize test fixing within their sprints?

If there is a less subjective way to ask this, I would be happy to accept any tips.

Answer 1

Make it so that's impossible to actually release anything without fixing the tests.

1. Fail the build if any tests fail.
2. Fail the build if any tests are ignored.
3. Fail the build if test coverage goes below a certain level (so people can't just delete tests to work around it).
4. Use the CI server to do your release builds, and only allow builds from the server's build drop to be promoted to UAT/staging/production/whatever.

The fact of the matter is, if your build is broken for more than about 15 minutes at a time (and that includes failing tests), then you aren't doing continuous integration.

The "nuclear option" is to have your source control server refuse commits/checkins from any user other than the one who broke the build. Obviously an admin needs to be able to override this temporarily if said person goes on holiday - but, if everybody knows that the whole team is screwed until they fix their tests, then they'll resolve it damn quick.

A good policy (which is even better when it's automated) is to revert the source to the last known stable commit after 15 minutes of the build failing. In other words, if you can't fix it, or don't know what caused the build or test to break, then revert it and work locally until it's resolved - never ever make other developers twiddle their thumbs while you grind away at a problem they don't care about.

P.S. If you already have a lot of tests failing, you can use a "trailing threshold" in CI. Set it up so that the build only fails if there are more test failures than last time. This, along with a coverage rule, will force developers to eventually improve the test situation if they want to be able to keep working.

P.P.S. I realize this might seem draconian to some, but it's all down your culture. If you get to a point where people just don't leave the build broken or tests failing (my team almost never does, although I occasionally have to remind them), then you don't need to continue with the strictest set of rules. Although IMO you should always fail the build on a broken unit test. Integration/browser tests can fail sometimes.

Answer 2

Units tests that fail are not the problem. They are a symptom.

The real problem is in the culture. You need to tread gently: here be dragons. You cannot change the culture by yourself, and being the squeaky wheel will, in the end, make you an outcast. Literally.

I suggest that if you try to find a senior person to champion the cause and lead the way. If that fails, try raising it in a general developers meeting, without pointing fingers or calling names. Another alternative is to take responsibility yourself for doing a proper job: just fix some more tests every time you do a check-in. Keep a chart on the wall showing how many tests fail over time. Others will see it: maybe they'll opt in.

There is no easy answer.