Why is design-by-contract not so popular compared to test-driven development?

StackOverflow https://stackoverflow.com/questions/481312

Question

You may think this question is like this question asked on StackOverflow earlier. But I am trying to look at things differently.

In TDD, we write tests that include different conditions, criteria, verification code. If a class passes all these tests we are good to go. It is a way of making sure that the class actually does what it's supposed to do and nothing else.

If you follow Bertrand Meyers' book Object Oriented Software Construction word by word, the class itself has internal and external contracts, so that it only does what its supposed to do and nothing else. No external tests required because the to code to ensure contract is followed is the part of the class.

Quick example to make things clear

TDD

  1. Create test to ensure that in all cases a value ranges from (0-100)

  2. Create a class containing a method that passes the test.

DBC

  1. Create a class, create a contract for that member var to range from (0-100), set contract for contract breach, define a method.

I personally like the DBC approach.

Is there a reason why pure DBC is not so popular? Is it the languages or tools or being Agile or is it just me who likes to have code responsible for itself?

If you think I am not thinking right, I would be more than willing to learn.

Was it helpful?

Solution

The main problem with DBC is that in the vast majority of cases, either the contract cannot be formally specified (at least not conveniently), or it cannot be checked with current static analysis tool. Until we get past this point for mainstream languages (not Eiffel), DBC will not give the kind of assurance that people need.

In TDD, tests are written by a human being based on the current natural-text specifications of the method that are (hopefully) well-documented. Thus, a human interprets correctness by writing the test and gets some assurance based on that interpretation.

If you read Sun's guide for writing JavaDocs, it says that the documentation should essentially lay out a contract sufficient to write a test plan. Hence, design by contract is not necessarily mutually exclusive with TDD.

OTHER TIPS

TDD and DbC are two different strategies. DbC permits fail-fast at runtime while TDD act "at compile time" (to be exact it add a new step right after the compilation to run the unit tests).

That's a big advantage of TDD over DbC : it allows to get earlier feedback. When you write code the TDD way, you get the code and its unit-tests at the same time, you can verify it "works" according to what you thought it should, which you encoded in the test. With DbC, you get code with embedded tests, but you still have to exercise it. IMO,this certainly is one reason that Dbc is not so popular.

Other advantages : TDD creates an automatic test suite that allow detecting (read preventing) regressions and make Refactoring safe, so that you can grow your design incrementally. DbC does not offer this possibility.

Now, using DbC to fail-fast can be very helpful, especially when your code interfaced other components or has to rely on external sources, in which case testing the contract can save you hours.

First of all, I am an Eiffel software engineer, so I can speak to the matter from experience.

The premise of TDD vs DbC is incorrect

The two technologies are not at odds with each other, but complementary to each other. The complement has to do with the placement of assertions and purpose.

The purpose of TDD has both components and scope. The basic components of TDD are boolean assertions and object feature (e.g. method) execution. The steps are simple:

  1. Create an object.
  2. Execute some code in a feature.
  3. Make assertions about the state of the data on the object.

Assertions that fail, fail the test. Passing all assertions is the goal.

Like TDD, the contracts of Design-by-Contract have purpose, scope, and components. While TDD is limited to unit-test-time, contracts can live through the entire SDLC (Software Development Life-cycle)! Within the scope of TDD, execution of object methods (features), will execute the contracts. In an Eiffel Studio Auto-test (TDD) setup, one creates an object, makes the call (just like TDD in other languages), but here is where likeness ends.

In Eiffel Studio with Auto-test and Eiffel code with contracts, the purpose changes somewhat. We want to test the Client-Supplier relationship. Our TDD code is pretending to be a Client of our Supplier method on its object. We create our objects and call the methods based on this purpose, and not just simplistic "TDD-ish method testing". Because the calls to our methods (features) have contracts, those contracts will execute as a part of our TDD-ish code in Auto-test. Because this is true, contract assertions (tests) that we place in our code do NOT have to appear in our TDD test code. Our job (as a programmer) is to simply ensure: A) The code + contracts are executed along all N-paths, and B) The code + contracts are executed using all reasonable data types and ranges.

There is perhaps more to write about the TDD-DbC complement relationship, but I won't be boorish with you on the matter. Suffice it to say that TDD and DbC are NOT at odds with other—not by a long shot!

The power of the contracts of DbC beyond where TDD can reach

Now, we can turn our attention to the power of the contracts of Design-by-Contract beyond where TDD can reach!

Contracts live in the code. They are not external to it, but internal. The most powerful bit (beyond their client-supplier contract relationship basis) about contracts is that the compiler is designed to know about them! They are NOT a bolt-on addition to Eiffel! Thus, they participate in every aspect of inheritance (both traditional vertical is-a inheritance and in lateral or horizontal Generics). Moreover, they reach to a place that TDD cannot reach—inside the method (feature).

While TDD can mimic pre-conditions and post-conditions with some ease, TDD cannot reach inside the code and perform loop-invariant contracts, nor can it do periodic spot-check "check" contracts along a block of code as it is executing. This is a powerful logical and qualitative paradigm, and a reality about how design-by-contract works.

Moreover, TDD cannot do class invariants but in the faintest of ways. I have tried my hardest to get my Auto-test code (which is really just Eiffel Studios version of applied-TDD) to do class invariant mimicry. It is not possible. To understand why you would have to know the in's-and-out's of how Eiffel class invariants work. So, for the moment, you will simply have to either take my word for it (or not) that TDD is incapable of this task, that DbC handles so easily, well, and elegantly!

The reach of DbC does not end with the above notions

We noted above that TDD lives at unit-test-time. Contracts, because they are applied in code under the supervision and control of the compiler, apply anywhere that the code can be executed:

  1. Workbench: you, as a programmer, are using the code to see it work (e.g. before TDD-time or in conjunction with TDD-time).

  2. Unit-test: your continuous integration testing, unit-testing, TDD, etc.

  3. Alpha-test: your initial test users will trip over contracts as they run the executable

  4. Beta-test: a wider audience of users will also trip over contracts.

  5. Production: the final executable (or production system) can have continual testing applied through contracts (TDD cannot).

In each of the situations above, one will find that one has control over just which contracts run and from what sources! You can selectively and fine-grainly turn on and off various forms of contracts and control with extreme precision where and when they are applied by the compiler!

And if all of this was not enough, contracts (by design) can do something that no TDD assertion can ever do: tell you where in the call-stack and which client-supplier relationship is broken, and why (which also immediately suggests how to fix it). Why is this true?

TDD assertions are designed to tell you about the results of a code-run (execution) after the fact. TDD assertion can only see as far as the current state of the method under examination. What TDD assertions cannot do from their position on the outside of the code-base is to tell you precisely which call failed and why! You see—your initial TDD call to some method will trigger that method. Many times, that method will call another, and another, and another—sometimes, as the call-stack winds up and down and hither and yon, there is a breakage: Something writes data wrong, does not write it at all, or writes it when it ought not.

TDD is like the police showing up to the crime scene after the murder has already happened. All they have left is forensic clues that they hope will lead them to a suspect and a conviction. But what if we could be there as the crime was taking place? That is the difference between the placement of TDD assertions and contract assertions. Contracts are there to catch the crime in progress and they point directly at the offender as it is committing the offense!

Recap

Let's recap.

  • TDD is not at odds with DbC.

  • It is a complement and a cooperative set of technologies, but with different functions and purposes, as well as tools to work with them.

  • Contract reach further and reveal more about your code when it breaks.

  • TDD is one form of catalyst for contracts to be executed.

At the end of the day: I want both! After reading all of this (if you survived), I hope you do as well.

Design-by-contract and test-driven development are not mutually exclusive.

Bertrand Meyer's book Object Oriented Software Construction, 2nd Edition doesn't say that you never make mistakes. Indeed, if you look at the chapter "When the contract is broken", it discusses what happens when a function fails to accomplish what its contract states.

The simple fact that you use the DbC technique doesn't make your code correct. Design-by-contract establishes well-defined rules for your code and its users, in the form of contracts. It's helpful, but you can always mess things up anyway, only that you'll probably notice earlier.

Test-driven development will check, from the outside world, black box style, that the public interface of your class behaves correctly.

I think it is best to use both methods in conjunction rather than just one or the other.

It has always seemed to me that fully enforcing a contract within the class and its methods themselves can be impractical.

For example, if a function says it will hash a string by some method and return the hashed string as output, how does the function enforce that the string was hashed correctly? Hash it again and see if they match? Seems silly. Reverse the hash to see if you get the original? Not possible. Rather, you need a set of test cases to ensure that the function behaves correctly.

On the other hand, if your particular implementation requires that your input data be of a certain size, then establishing a contract and enforcing it in your code seems like the best approach.

In my mind TDD is more "inductive". You start with examples (test cases) and your code embodies the general solution to those examples.

DBC seems more "deductive", after gathering requirements you determine object behavior and contracts. You then code the specific implementation of those contracts.

Writing contracts is somewhat difficult, more so than tests that are concrete examples of behavior, this may be part of the reason TDD is more popular than DBC.

I've used both in the past and found DBC-style less "intrusive". The driver for DBC may be regular application running. For Unit Tests you have to take care of setup because you expect (validate) some responses. For DBC you don't have to. Rules are written in data-independent manner, so no need to setup and mocking around.

More on my experiences with DBC/Python: http://blog.aplikacja.info/2012/04/classic-testing-vs-design-by-contract/

I see Design By Contract as a specification for success/failure in ALL cases, whereas Test Driven Development targets ONE specific case. If the TDD case succeeds, then it is assumed a function is doing it's job, but it doesn't take into account other cases that could cause it to fail.

Design By Contract on the other hand doesn't necessary guarantee the desired answer, only that the answer is "correct." For example, if a function returns is supposed to return a non-null string, the only thing you can assume in the ENSURE is that it will not be null.

But maybe it doesn't not return the string that was expected. There is no way for a contract to be able to determine that, only a Test can show that it was performing according to the specification.

So the two are complementary.

Greg

I see no reason why both cannot co-exist. It is wonderful to look at a method and know at a glance exactly what the contract is. It is also wonderful to know that I can run my unit tests and know that nothing was broken with my last change. The two techniques are not mutually exclusive. Why design by contract is not more popular is a mystery.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top