TDD Duplicate Testing on Related Classes

Question

In following the principle of testing only the exported functions on a package (using Go - or for others languages, the public functions on a class), I'm running into a scenario where related packages are causing the code base to be tested multiple times.

For example, I have a Preprocessor, which calls a Parser package, which calls a Scanner package which calls a Lexer package.

Preprocessor -> Parser -> Scanner -> Lexer

The results bubble up to the previous package and the final results are compiled in the Preprocessor. No package below the Preprocessor is called by any other part of the app.

Each package has tests, which means the lexer is tested 4x, the scanner 3x, the parser 2x and the preprocessor 1x.

I believe each feature should be its own separate package because if I combined them all into a single Preprocessor package, the parser, scanner and lexer don't need a public API, yet the functionality is too complex not to test individually as it would be difficult to track down where bugs were occurring.

Is this type of duplicate testing recommended or is it a sign that the packages are too related to be separated? Or, is it a sign that the lower level packages are doing too much and I should extract more business logic into the Preprocessor package such that it could call each of the lower level packages, instead of having their requests chained together?

Answer 1

A technique to avoid this - if it is in fact a problem for your project - is to use mocks. The Scanner, for instance, would be tested with a mock Lexer; the Parser with a mock Scanner, etc. In this way your actual Lexer is only exercised by the Lexer tests.