Testing iterative code using mocks - does it make sense, how?

Question

I want to test an algorithm using mocks. The algorithm - in the current implementation - iterates over a container class in multiple passes and set()s and get()s values from it. The test's purpose is to verify the final result, stored in the container. The final value depends on values read and written in between the passes. e.g. the value of any element may change multiple times until the algorithm is finished and, most-likely, its value as result of iteration n will depend on its value after iteration n-1.

I like the idea of mocks and I would love to use them in the scenario described above since it would allow me to verify erroneous behavior of the algorithm once it occurs, not only when the computation is finished. However, I'm not sure whether this would actually be a good idea because I would then have to tie the expectations for the mock real close to the current implementation (e.g, "expect get(element n) and return x, then set(element n, value x+1), another get(n) and return x+1, then expect set(n, x+2) etc.").

Although allowing me to verify that intermediate values are as expected, I think such expectations would contradict the purpose of the test (verify that the algorithm computes the correct final value) and likely the test would fail if the implementation changes, regardless of the correctness of the final value.

Now my question: am I missing something? Is there nevertheless a good way to use mocks in this scenario? or does it simply make no sense to use them here? How do others deal with this problem?

Final remark: I'm talking about testing c++ code and using googlemock, if that makes any difference to your answer.

p.s.: I checked google and articles here (especially Mocking iterative behaviour - only tackles the problem of increasing a return value), however, I didn't find anything close to my problem.

Answer 1

I would say that if the container is slow in some way or it has side-effects that mean you cannot read its values without disturbing it, then you should use a mock.

Otherwise, using a mock is a waste of time. Would you use a mocked version of std::vector? I wouldn't; it would be silly.

With your unit testing, if you cannot test all of the internal state of your algorithm through various public parameters, then those states don't actually matter. They can never come up in real use. So as long as you get the correct final answer from your algorithm for each set of input parameters, I would say things are working well.

Answer 2

Create your unit test for the final output of the algorithm. You want your automated testing to verify the expected result because that is what other parts of the program will utilize.

As far as testing the individual steps within the algorithm code, this is more a job for stepping through with a debugger -- not for automated tests. Going through the internal workings of the algorithm should be a one-time thing, and once you have it right there's no need to keep testing individual steps within it.

Answer 3

Unit tests would be more applicable on the smaller parts that compose the algorithm.

That being said, unit test tools are very helpful for developing algorithms that way. They aren't bad at all to use, just don't hold on to them if they aren't valid anymore. Usually you wouldn't test each iteration in an integration test, you'd just test the results. If it is helpful to develop the algorithm that way though, go for it.

You are right about the mocks, you aren't really testing a lot. They can be useful if you want to control some of the inputs though. Many times I will permute my inputs in exhaustive ways when I have black boxes that I cannot control. These sorts of tests run too long, though. I will generally comment them out entirely or partially when they go into source control.