How to do documentation for code and why is software (often) poorly documented?

Question

There are some good examples of well-documented code out there, such as java api. But, a lot of code in public projects such as git and internal projects of companies is poorly documented and not very newcomer friendly.

In all my software development stints, I have had to deal with poorly documented code. I noticed the following things -

1. Little or no comments in code.
2. Method and variable names are not self describing.
3. There is little or no documentation for how the code fits into the system or business processes.
4. Hiring bad developers or not mentoring the good ones . They can't write simple and clean code. Hence its difficult or impossible for anyone, including the developer to document the code.

As a result, I have had to go through a lot of code and talk to many people to learn things. I feel this wastes everyone's time. It also creates the need for KT/Knowledge transfer sessions for newcomers to a project.

I learned that documentation is not given the attention it deserves because of the following reasons:

1. Laziness.
2. Developers don't like to do anything but code.
3. Job security. (If no one can understand your code easily, then you might not be easily replaceable.)
4. Difficult deadlines leave little time to document.

So, I am wondering if there is a way to encourage and enforce good documentation practices in a company or project. What are the strategies to be used for creating decent documentation for the systems and code of any project, regardless of its complexity ? Are there any good examples of when minimal or no documentation is needed ?

IMHO, I feel that we should have a documentation review after a project is delivered. If it is not simple, concise, illustrative and user friendly, the developer or technical documentation engineer own the responsibility for it and be made to fix it. I neither expect people to make reams of documentation, not hope that it will be user friendly like the head first books, but I expect it to eliminate the need for hours of analysis and wasteful KT sessions.

Is there a way to end or alleviate this madness ? "Document driven development" perhaps ?

Answer 1

How to document code?

You already have a hint: look at how Java API is documented.

More generally, there is no unique set of rules which apply to every project. When I work on business-critical large-scale projects, the documentation has nothing to do with the one I would write for a small open source library, which, in turn, has nothing to do with the documentation of my medium-scale personal project.

Why many open source projects are not documented well?

Because most open source projects are made by people who contribute to those projects because it's fun. Most programmers and developers consider that writing documentation is not fun enough to be done for free.

Why many closed-source projects are not documented well?

Because it costs a huge amount of money to (1) write good documentation and to (2) maintain it.

1. The immediate cost (cost of writing the documentation) is clearly visible to the stakeholders: if your team asks to spend the next two months documenting the project, it's two additional months of salary to pay.

2. The long term cost (cost of maintaining the documentation) becomes noticeable pretty easy too to the managers, and is often the first target when they must lower the cost or shorten the delays. This causes an additional problem of outdated documentation which quickly becomes useless, and is extremely expensive to update.

3. The long term savings (savings from not having to waste a few days exploring the legacy code just to understand a basic thing which should have been documented years ago) are, on the other hand, difficult to measure, which confirms the feeling of some managers that writing and maintaining documentation is a waste of time.

What I often observe is that:

1. At the beginning, the team is willing to document a lot.

2. Over time, pressure of deadlines and lack of interest make it more and more difficult to maintain the documentation.

3. A few months later, a new person who joins the project practically can't use the documentation, because it doesn't correspond at all to the actual system.

4. Noticing that, management blames developers for not maintaining the documentation; developers ask to spend a few weeks updating it.

   • If management grants a few weeks for that, the cycle repeats.

   • If management refuses, based on previous experience, it only increases the bad experience, since the product lacks documentation, but a few months were spent writing and maintaining it.

Documentation should be a continuous process, just like testing. Spend a week simply coding a few thousands of LOC, and getting back to tests and documentation is very, very painful.

How to encourage the team to write documentation?

Similarly to the ways to encourage people to write clean code, to do regular refactoring, to use design patterns or to add enough unit tests.

• Lead by example. If you write good documentation, your pairs might start doing it too.

• Do systematic code reviews, including formal code reviews targeted at inspecting the documentation.

• If some members of the team are particularly antipathetic to good documentation (or documentation at all), discuss the subject with them privately, to understand what are the impediments which prevent them from writing better documentation. If they blame the lack of time, you see the source of the problems.

• Make the presence or the lack of documentation measurable for a few weeks or months, but don't focus on that. For example, you may measure the number of lines of comments per LOC, but don't make it a permanent measure, otherwise, developers will start writing long but meaningless comments just to get rid of low scores.

• Use gamification. This comes together with the previous point.

• Use positive/negative reinforcement.

• (See the comment by SJuan76) Treat the lack of comments as errors. For example, in Visual Studio, you can check an option to generate XML documentation. If you also check that all warnings are treated as errors, the lack of a comment at the top of a class or a method will halt the compilation.

  As for the three previous points, this one should be used with caution. I used it for a while with a particularly tough team of beginner programmers, and it ended up with StyleCop-compliant comments like that:

    /// <summary>
    /// Gets or sets the PrimaryHandling.
    /// </summary>
    public Workflow PrimaryHandling { get; set; }

which were, hm..., not particularly helpful.

Remember: nothing automated can help you pinpoint bad comments when programmers wants to screw with you. Only code reviews and other human tasks will help.

Are there any good examples of when minimal or no documentation is needed?

Documentation explaining the architecture and the design is not needed:

• For a prototype,

• For a personal project written in a few hours to accomplish a task, while being pretty sure this project won't be maintained any longer,

• For any project where it is obvious, given the small size of it, coupled with the particularly clean code, that you will spend more time writing documentation than all the future maintainers exploring the code.

In-code documentation (code comments) is not needed according to some developers when the code is self-documenting. For them, the presence of comments is, except in the rare cases, not a good sign, but a sign that the code wasn't refactored enough to be clear without the need for comments.

I feel that we should have a documentation review after a project is delivered.

If your project is delivered at least once per week, it's the way to go. If your project is not agile and is delivered at the intervals of six months, then do more regular reviews.

Answer 2

I think you should make a distinction between comments and documentation. While comments are descriptive, they lack consistancy, they are scattered all over the code. Comments should never compensate for code which is not self-describing enough, instead it should hint other programmers at tricky parts.

Whether code should be documented depends on the scale of the project. Surely there are people who believe that everything should be documented, and it seems easy to justify that thought because who would dare to oppose documenting knowledge? Fortunately software development is not science, and the world doesn't collapse if the details behind your little program become obscure. Now concerning a professional software for use by many developers, yes obviously you should document your code. But if you are in the position to code on that level, then you would obviously know that.

tl;dr

If you're asking for every project to be well documented, then you're asking too much.