Two Papers on Errors

As regular readers will have gathered, I think we can learn a lot of useful things by studying programmers' mistakes and the inadequacies of the tools we have for detecting, reporting, handling, and correcting them. The first of these papers, Barbosa2014, presents an empirical taxonomy of the ways exceptions are mis-handled, the first two entries in which have multiple sub-cases:

1. Information swallowed
   • Uninformative or wrong error message
   • Swallowed exception: the exception is caught and ignored, often just to silence compiler checks
   • Suppressed exception: the original exception is hidden by another exception raised in the same context
   • Missing log message: the exception is caught and handled but there's no record of it happening
   • Destructive remapping: an exception is caught, mapped to another with information loss, and rethrown
   • Uninformative generic type throws: the code throws a generic (and uninformative) exception rather than a more specific exception
2. Improper continuation of execution
   • Missing throwing condition: a condition that should have led to an exception being thrown wasn't checked (e.g., it's missing from a multi-part logical test)
   • Wrong context configuration: bad exception handling leaves the system in an inconsistent state (e.g., without access to an external resource)
   • Wrong location of execution resumption: the statements after the catch shouldn't be executed if an exception occurred
   • Missing termination action: actions that should always take place whether there was an exception or not only take place if an exception was thrown
3. Resource leak
4. Uncaught exception
5. Overly-generic catch block
6. Premature termination
7. Throwing wrong type
8. Overly protective try block
9. Exceptional loop break
10. Excessive throwing conditions

I don't know any linting tools that check for cases like these, but this list would be a great basis for a code review checklist (or a starting point for exercises in a code review training course).

At 41 pages, Coelho2016 is harder to summarize, but its results are just as valuable. The two main questions its authors set out to answer are:

1. Can the information available in exception stack traces reveal exception handling bug hazards in both the Android applications and framework?
2. How do developers deal with the exception handling code in Android apps and what are the developers' perspectives about the main bug hazards found during the mining study?

Unsurprisingly, null pointer exceptions are the most common source of problems. Depressingly, the authors found that runtime exceptions are often not documented; in particular, they found checked exceptions thrown by native methods that were not declared in the interface of the wrapper methods, which would be a debugging nightmare.

One thing I particularly liked in this paper was its shift from analyzing code to surveying and interviewing developers to answer its second research question. Far too many papers scrape GitHub and throw the results into a machine learning package without checking whether the data, the questions, or the findings make any sense to the people the work is supposedly for. Asking programmers "Does this make sense?" and "Is it consistent with your experience?" is a necessary and welcome corrective.