Struggling with cyclical dependencies in unit tests

Question

I'm trying to practice TDD, by using it to develop a simple like Bit Vector. I happen to be using Swift, but this is a language-agnostic question.

My BitVector is a struct that stores a single UInt64, and presents an API over it that lets you treat it like a collection. The details don't matter much, but it's pretty simple. The high 57 bits are storage bits, and the lower 6 bits are "count" bits, which tells you how many of the storage bits actually store a contained value.

So far, I have a handful of very simple capabilities:

1. An initializer that constructs empty bit vectors
2. A count property of type Int
3. An isEmpty property of type Bool
4. An equality operator (==). NB: this is a value-equality operator akin to Object.equals() in Java, not a reference equality operator like == in Java.

I'm running into a bunch of cyclical dependancies:

1. The unit test that tests my initializer need to verify that the newly constructed BitVector. It can do so in one of 3 ways:

   1. Check bv.count == 0
   2. Check bv.isEmpty == true
   3. Check that bv == knownEmptyBitVector

   Method 1 relies on count, method 2 relies on isEmpty (which itself relies on count, so there's no point using it), method 3 relies on ==. In any case, I can't test my initializer in isolation.

2. The test for count needs to operate on something, which inevitably tests my initializer(s)

3. The implementation of isEmpty relies on count

4. The implementation of == relies on count.

I was able to partly solve this problem by introducing a private API that constructs a BitVector from an existing bit pattern (as a UInt64). This allowed me to initialize values without testing any other initializers, so that I could "boot strap" my way up.

For my unit tests to truly be unit tests, I find myself doing a bunch of hacks, which complicate my prod and test code substantially.

How exactly do you get around these sorts of issues?

Answer 1

You're worrying about implementation details too much.

It doesn't matter that in your current implementation, isEmpty relies on count (or whatever other relationships you might have): all you should be caring about is the public interface. For example, you can have three tests:

• That a newly initialized object has count == 0.
• That a newly initialized object has isEmpty == true
• That a newly initialized object equals the known empty object.

These are all valid tests, and become especially important if you ever decide to refactor the internals of your class so that isEmpty has a different implementation that doesn't rely on count - so long as your tests all still pass, you know you haven't regressed anything.

Similar stuff applies to your other points - remember to test the public interface, not your internal implementation. You may find TDD useful here, as you'd then be writing the tests you need for isEmpty before you'd written any implementation for it at all.

Answer 2

How exactly do you get around these sorts of issues?

You revise your thinking on what a "unit test" is.

An object that manages a mutable data in memory is fundamentally a state machine. So any valuable use case is going to, at a minimum, invoke a method to put information into the object, and invoke a method to read a copy of information out of the object. In the interesting use cases, you are also going to be invoking additional methods that change the data structure.

In practice, this often looks like

// GIVEN
obj = new Object(...)

// THEN
assert object.read(...)

or

// GIVEN
obj = new Object(...)

// WHEN
object.change(...)

// THEN
assert object.read(...)

The "unit test" terminology -- well, it has a long history of not being very good.

I call them unit tests, but they don't match the accepted definition of unit tests very well -- Kent Beck, Test Driven Development by Example

Kent wrote the first version of SUnit in 1994, the port to JUnit was in 1998, the first draft of the TDD book was early 2002. The confusion had a lot of time to spread.

The key idea of these tests (more accurately called "programmer tests" or "developer tests") is that the tests are isolated from each other. The tests don't share any mutable data structures, so they can be run concurrently. There are no worries that the tests must be run in a specific order to correctly measure the solution.

The primary use case for these tests is that they are run by the programmer between edits to her own source code. If you are performing the red green refactor protocol, an unexpected RED always indicates a fault in your last edit; you revert that change, verify that the tests are GREEN, and try again. There isn't a lot of advantage in trying to invest in a design where each and every possible bug is caught by only one test.

Of course, is a merge introduces a fault, then finding that fault is no longer trivial. There are various steps you can take to ensure that faults are easy to localize. See

• Parnas: On the criteria to be used in decomposing systems into modules
• Rainsberger: Integrated tests are a scam
• Shore: Testing Without Mocks: A Pattern Language

Answer 3

In general (even if not using TDD) you should strive to write tests as much as possible while pretending you don't know how it is implemented.

If you're actually doing TDD that should already be the case. Your tests are an executable specification of the program.

How the call graph looks underneath the tests is irrelevant, as long as the tests themselves are sensible and well maintained.

I think your problem is your understanding of TDD.

Your problem in my opinion is that you are "mixing" your TDD personas. Your "test", "code", and "refactor" personas operate completely independently of each other, ideally. In particular your coding and refactoring personas have no obligations to the tests other than to make/keep them running green.

Sure, in principle, it would be best if all tests were orthogonal and independent of each other. But that is not a concern of your other two TDD personas, and it is definitely not a strict or even necessarily realistic hard requirement of your tests. Basically: Don't throw out your common sense feelings about code quality to try to fulfill a requirement that nobody is asking of you.