Unit testing and other forms of testing

Question

This may be a silly question but if I have good unit test coverage, does that mean I can reduce the amount of, or remove completely, functional and integration testing. When people talk about having a continuous deployment process in place, does that mean automated unit tests are the only form of testing done and the code is deployed to production straight away?

Answer 1

Strangely enough the best answer to this question is the Single Responsibility Principle.

But that only works if you think of it in the revised way Uncle Bob explains it today. If you think of SRP as simply being "Do one thing" this answer likely makes no sense.

The final version of the SRP is:

A module should be responsible to one, and only one, actor

Robert C. Martin - Clean Architecture

By actor he means a group of people performing a single role that have some interest in the code. These people should be the only people able to force you to change this code. If your code answers to two groups of people performing two different roles then your code is serving two masters.

Which reminds me of magic numbers. Yeah I know. Hang on.

When you apply the Don't Repeat Yourself concept you refactor out duplication. Mostly this is a good thing but some duplication exists for a reason. I could replace every constant variable that is 3 with 3. Why don't I? Because the variable names represent different concepts. Structurally it's still just a 3 but we want more than structure. This same concept also exists in functions, classes, modules, and tests.

What you're talking about is pretty much applying DRY to your different kinds of testing. This is fine but don't make the classic mistake people make when they over apply DRY. It isn't structural duplication we want to eliminate. We want each decision to live in only one place. If two separate decisions happen to look identical don't squish them together and pretend they are the same decision. It's ok to have two constant vars that both equal 3. It's ok to have two methods that hold the same code. It's ok to have two kinds of tests testing the same thing. So long as the idea is different. So long as the actor the code is responsible to is different.

I know, it's hard. It's so much easier to judge based on what the code looks like but what we really care about is what it's for. I last talked about this idea here.

So don't tell me you're testing is complete just because you have complete code coverage. That isn't the main point of testing. It never will be. It's supposed to reassure us that the code base does what people are expecting it to do. And many different people have many different expectations.

Now all that said, if people are hanging on to old tests because of concerns that are better addressed by other tests then dump the old tests. Otherwise things become cluttered.

Answer 2

Unit tests and integration tests exists for different reasons.

Unit tests are written by programmers to test only one function (responsibility). Here you can apply SRP rule. Tests should run fast and must be easy to execute. They are running frequently for example after some bug fix in code.

Integration tests exists to execute different pieces of system to see if they work well together. They are using external resources like database instance so they are running longer and could be much more difficult to start. Also integration tests demonstrate how the system work.

So you can have unit tests and integration tests both in your solution because they exists for different purposes.

Answer 3

We can pontificate on what is and isn't a unit test for all eternity, seeking to answer this question.

Or, we can have a good laugh looking at the images and watching the videos that come up when searching for two unit tests no integration tests. The images are funny, but they also bring home a serious message, no you should not "reduce the amount of, or remove completely, functional and integration testing" just because you have some unit tests. And also, no amount of automated functional and integration checking will ever remove the need for a person to carry out proper exploratory testing either.

Answer 4

Functional and integration testing cannot be minimized, even with the best of all possible developer/unit-testings teams. The new code must be executed in a real-life (or as close as possible to it) flow along side all of the other code that runs in a system. Unfortunately even the best development teams still eliminate (intentionally or accidentally) real-world complications for the sake of unit development. These complications can expose errors in product.

Answer 5

There are a number of aspects to your question.

Firstly, unit tests are there to prove the correctness of the code. Test coverage simply illustrates how much of the code is executed. Having good code coverage does not mean you have well written unit tests.

As for whether there are tests that can be removed - that would rather depend on whether there is any duplication. Only the developers would be able to answer this although certain tools can help.

Continuous Deployment (CD) simply means that the code is pushed out once it has been deployed. This has nothing to do with testing although it would be seriously inert of the build manager to allow this to happen if the tests didn't pass!

As for whether this happens only if this unit tests pass, this again is a question for the build manager. Some only allow unit tests to run while others run integration tests too. It usually depends on the build throughput. If builds are constantly queuing because integration tests are running, they may be inclined to set the build to only run unit tests or have a separate build that runs the integration tests less frequently (perhaps overnight or weekly etc).