The Documentation Challenge

July 28, 2009 @ 21:58 | In Programming | 2 Comments | twitter rss digg
Image with lots of books

Documentation is the part of the development process where usually less time and money is invested. Few resources implies a poor infrastructure for something that, as the project gets bigger and bigger, becomes a very important part of the overall process.

I want to share in this post the approach we are following for a project I have been involved with for over two years. This project (I will be able to give details about it very soon) is mainly a framework composed of libraries to be used by other companies. Basically, a work from developers to developers, clearly a context where documentation becomes very important.

The main objectives we tried to achieve with this architecture:

  • Documentation must be located in the code repository. This means that each development branch has its own documentation that is integrated in the same way that the code.
  • Documentation generation must be part of the building process. We use our build pipeline for everything: compile versions, test versions, compile data packages, generate installers, distribute symbols and, of course, build documentation. This means that the source of the documentation must be a very basic representation, easy to edit, that later is converted to a pdf, doc, html by the building process. This point implies that the documentation can be easily distributed to clients or even generated by themselves.
  • Access to current documentation must be integrated into the development environment. In the same way I can review bugs in the bugtracker, stories in the task manager and browse the source code I must be able to read the current documentation from a web browser without having to rebuilt it manually.

We make the distinction between two kind of documentation: API Documentation and Technical Articles.

API documentation are those documents where class usage is described: description for each function, for each parameter, enumerations, etc. Doxygen seems the perfect fit for this purpose. But after several projects using it we opted for not using it in this project because we considered it a waste of time. Doxygen may be really useful to generate documentation for a library that is distributed without sources but one of the pillars of our development philosophy is to always provide source code to clients. This way, they can choose how to compile it: one dll, several dlls, one lib… and tweak the different configurations. Another good rule that we follow is that code must document itself. Doxygen would force us to write code like this:

class MemoryManager: public MemoryAllocator
    /// Allocates memory on a specified alignment boundary
    /// \param size Size in bytes of the block requested by the user
    /// \param alignment The alignment value, which must be an integer power of 2
    /// \return A pointer to the new allocated block of memory
    /// \remarks Thread-safe
    virtual void* AlignedAlloc(size_t size, size_t alignment) = 0;

Quite redundant code, don’t you think? Obviously AlignedAlloc() allocates memory that is aligned on a boundary, I do not need a comment for that! The same for the function parameters. So we ended up writing documentation for only the parts that are really confusing or need a clarification (for example, the fact that AlignedAlloc() is thread-safe) and using the source code as the documentation itself for both the interface and the implementation. We augment this documentation by using unit tests that serve as example usage for each class. We know this decision is a controversial one and may be in the future we return back to using Doxygen or something similar. But for now, it is working fine.

A technical article is like a manual that describes the architecture and philosophy of a specific part of the framework. They provide information to a deeper level than the mere description of classes provided by the API documentation. We usually link to technical articles from API documents.

Although a wiki seems an ideal candidate for writing technical articles we rejected it because it didn’t satisfy the points enumerated above. Years ago and inspired by the Gentoo Documentation Project I used DocBook for a similar purpose. I wrote about it in a previous article: Documentation using XML. And although the results were quite satisfactory I didn’t want to return to that solution, mainly because a XML format is hard to write, edit, maintain, etc. I think we found a better solution: reStructuredText. reStructuredText is part of the DocUtils suite and from very simple what-you-see-is-what-you-get plaintext markup it generates quite good looking documents. Another bonus feature that finally inclined the balance towards reStructuredText is that it perfectly integrates with Trac which we have been using since the beginning. Trac is a suite that integrates wiki, bugtracking, source code browsing and task management in a single web program. I like to see trac as a free alternative to FogBuz. Trac supports both its own wiki format and reStructuredText allowing us to integrate the documentation located in the repository as part of our wiki infrastructure satisfying the last point mention above. Time will say if this infrastructure will suffice.

Want to comment how do you solve the documentation issues in your project? Please do so. Comments are open. Thanks for reading.

  1. Yay for keeping documentation to a minimum and using the code and unit tests as the main way of documenting things. It’s not that documentation gets out of date (which it does all the time), the main problem is that most people don’t read the documentation. A failing unit test, on the other hand, it’s hard to ignore :-)

    Comment by Noel
    July 29, 2009 @ 11:27 #

  2. Doxygen itself also has quite simple yet powerful markup language, so I think it is very good for writing technical articles, not only documenting code.

    Comment by Reg
    August 2, 2009 @ 16:24 #

Sat, 23 Aug 2014 19:47:16 +0000 / 27 queries. 1.190 seconds / 2 Users Online

gentoo link wordpress link apache link PHP link

Theme modified from Pool theme. Valid XHTML and CSS