Syntax, Predicates, Idioms—What Really Affects Code Complexity?

Reviewed by Greg Wilson / 2021-09-21
Keywords: Program Comprehension, Programming Languages

Are some programming constructs intrinsically more difficult to read and write than others? Ajami2018 explored this question by having over 200 professional programmers look at code snippets that did similar things in different ways. Their findings include:

  • for loops are harder for people to understand than if statements,
  • loops that count down are (slightly) harder to understand than loops that count up,
  • time taken and error rate don't necessarily correlate, which means that the former cannot be used as a proxy for the latter.

Like all results, these need to be interpreted carefully: for example, the authors' tests used C-style for loops, which are syntactically much more complex than the for loops of languages like Python. However, the paper spells out the statistical methods used in welcome detail, and is careful to account for learning effects and other confounding factors. (Sentences like, "More ranges also led to higher error rates, but this was not significant after Bonferroni correction." make me very happy.)

I am a big fan of work like this, which I first met through papers by Andreas Stefik and his team on the readability and learnability of programming languages (see these three posts for examples). It is very welcome to see other research groups doing studies like these as well, and I hope one day that things like Python Enhancement Proposals will routinely include studies like these. Among other things, it would be a great way to build bridges between researchers and practitioners.

Ajami2018 Shulamyt Ajami, Yonatan Woodbridge, and Dror G. Feitelson: "Syntax, predicates, idioms—what really affects code complexity?". Empirical Software Engineering, 24(1), 2018, 10.1007/s10664-018-9628-3. (Reviewed 2021-09-21.)

Program comprehension concerns the ability to understand code written by others. But not all code is the same. We use an experimental platform fashioned as an online game-like environment to measure how quickly and accurately 220 professional programmers can interpret code snippets with similar functionality but different structures; snippets that take longer to understand or produce more errors are considered harder. The results indicate, inter alia, that for loops are significantly harder than ifs, that some but not all negations make a predicate harder, and that loops counting down are slightly harder than loops counting up. This demonstrates how the effect of syntactic structures, different ways to express predicates, and the use of known idioms can be measured empirically, and that syntactic structures are not necessarily the most important factor. We also found that the metrics of time to understanding and errors made are not necessarily equivalent. Thus loops counting down took slightly longer, but loops with unusual bounds caused many more errors. By amassing many more empirical results like these it may be possible to derive better code complexity metrics than we have today, and also to better appreciate their limitations.