To Type or Not to Type: Quantifying Detectable Bugs in JavaScript

Way back in 2012 we reviewed Hanenberg2010, which found that (under very specific and carefully explained circumstances) the use of static typing did not make programmers more productive than the use of dynamic typing. However, in a later paper (Hanenberg2013), researchers looked at, "…whether static type systems improve the maintainability of software systems, in terms of understanding undocumented code, fixing type errors, and fixing semantic errors" and found that, "…static types are indeed beneficial to these activities, except when fixing semantic errors." Digging deeper, they found that, "…developers using a dynamic type system tend to look at different files more frequently when doing programming tasks—which is a potential reason for the observed differences in time."

The authors of Gao2017 took a different approach to tackle the same underlying question. They look at bugs in production JavaScript systems, then went back and added type annotations to the buggy code to see whether static typing would have caught the error. The answer was "yes" in 15% of cases, which is either high or low depending on your perspective. Their experiment doesn't tell us whether static typing is cost-effective (e.g., whether any extra time taken to declare types would find more bugs if it was spent writing unit tests), but it's still a very useful answer, and it would be great to see this experiment repeated for another language like Python.

Addendum: one other interesting study of static typing is Fischer2015, which looked at whether the benefits of static typing are due to better code completion in the IDE. The answer appears to be "no", i.e., whatever effect static typing has is because of the typing itself, not because it helps a modern development tool do the right thing when you press the tab key. Teasing apart possible confounding factors like this is central to any modern research program; if anyone knows of an up-to-date summary of related research we'd be grateful for pointers.