Question

Continuous Delivery sounds good, but my years of software development experience suggest that in practice it can't work.

(Edit: To make it clear, I always have lots of tests running automatically. My question is about how to get the confidence to deliver on each checkin, which I understand is the full form of CD. The alternative is not year-long cycles. It is iterations every week (which some might consider still CD if done correctly), two weeks, or month; including an old-fashioned QA at the end of each one, supplementing the automated tests.)

  • Full test coverage is impossible. You have to put in lots of time -- and time is money -- for every little thing. This is valuable, but the time could be spent contributing to quality in other ways.
  • Some things are hard to test automatically. E.g. GUI. Even Selenium won't tell you if your GUI is wonky. Database access is hard to test without bulky fixtures, and even that won't cover weird corner cases in your data storage. Likewise security and many other things. Only business-layer code is effectively unit-testable.
  • Even in the business layer, most code out there is not simple functions whose arguments and return values can be easily isolated for test purposes. You can spend lots of time building mock objects, which might not correspond to the real implementations.
  • Integration/functional tests supplement unit tests, but these take a lot of time to run because they usually involve reinitializing the entire system on each test. (If you don't reinitialize, the test environment is inconsistent.)
  • Refactoring or any other changes break lots of tests. You spend lots of time fixing them. If it's a matter of validating meaningful spec changes, that's fine, but often tests break because of meaningless low-level implementation details, not stuff that really provides important information. Often the tweaking is focused on reworking the internals of the test, not on truly checking the functionality that is being tested.
  • Field reports on bugs cannot easily be matched with the precise micro-version of the code.

No correct solution

Licensed under: CC-BY-SA with attribution
scroll top