Generating Unit Tests for Documentation

This paper describes a tool called DScribe, which generates documentation and unit tests simultaneously from templates like this:

  (doc)  If no argument is provided, the default value for the
         arguments of $method$ is $defVal$.
  (test) @Test
         public void test$method$_defaultArgument() {
           assertEquals( $method$() , $method$($defVal$) );
         }
  

It's a clever extension of things like Python's doctest, but is it actually useful? To find out, the authors studied the inconsistencies between source code, unit tests, and documentation in exceptions in the Apache Commons IO library. They found that:

Among the sampled methods of Commons IO, only 15% of the specifications related to thrown exceptions (ESU) are both tested and documented. The remaining 85% of ESUs represent inconsistent information that would require a non-trivial amount of effort to manually fix. DScribe can generate the tests and documentation for 97% of these ESUs, using only five templates.

They also found that:

Approximately 42% of the unit tests...capture information that would be relevant to document, but that information is documented for only 52% of these tests. Developers can use DScribe to generate documentation from these tests more consistently and efficiently, especially for repetitive specifications.

I don't know if I'll ever use DScribe itself, but I would be surprised if the ideas in it don't show up in other tools in years to come: as more languages edge toward literate programming, putting all the information about a piece of software in a single place becomes more feasible, and I look forward to seeing what other clever ideas emerge.