Commenting source code: is it worth it for small programming tasks?

Reviewed by Greg Wilson / 2021-09-06
Keywords: Commenting, Program Comprehension

This paper from 2018 builds on previous studies of the effects of comments in code by tackling three questions:

  1. Is there a significant difference between the types of commented code and the correctness or time of task solving, respectively?
  2. Is there a significant difference between novice and professional programmers in the correctness or time of task solving for differently commented code types?
  3. Are the programmers' self-assessments of comments coherent with the observed results?

The authors tackled the first two questions with an experiment involving 277 programmers, and the third with a questionnaire answered by 157 people. The experiment involved using existing code, fixing bugs, and extending code; programmers didn't know the goal was to assess the impact of comments (in order to avoid bias), and were allowed to use their preferred IDE. Their conclusions?

  • "Among all tasks and participants, we cannot see a significant influence of the differently commented code types on the correctness at all. Instead…the proportions of correct and wrong answers are mainly driven by the tasks themselves."
  • "…the differences between execution times of submissions with documentation comments is significantly higher than those of the other two code types [note: implementation comments and no comments at all]…"
  • "…professionals seem to perform especially well [compared to novices] when no comments are provided."
  • "The subjective assessment of our participants indicates that they see a more positive impact of comments than we found."

Coincidentally, I also just found Daniel Stenberg's blog post Making World-class Docs Takes Effort, which claims that the best documentation is not extracted from code, but written separately. I haven't found a study yet that compares in-code comments to external explanatory documentation, but I think it would be a useful and interesting topic.

Nielebock2018 Sebastian Nielebock, Dariusz Krolikowski, Jacob Krüger, Thomas Leich, and Frank Ortmeier: "Commenting source code: is it worth it for small programming tasks?". Empirical Software Engineering, 24(3), 2018, 10.1007/s10664-018-9664-z.

Maintaining a program is a time-consuming and expensive task in software engineering. Consequently, several approaches have been proposed to improve the comprehensibility of source code. One of such approaches are comments in the code that enable developers to explain the program with their own words or predefined tags. Some empirical studies indicate benefits of comments in certain situations, while others find no benefits at all. Thus, the real effect of comments on software development remains uncertain. In this article, we describe an experiment in which 277 participants, mainly professional software developers, performed small programming tasks on differently commented code. Based on quantitative and qualitative feedback, we i) partly replicate previous studies, ii) investigate performances of differently experienced participants when confronted with varying types of comments, and iii) discuss the opinions of developers on comments. Our results indicate that comments seem to be considered more important in previous studies and by our participants than they are for small programming tasks. While other mechanisms, such as proper identifiers, are considered more helpful by our participants, they also emphasize the necessity of comments in certain situations.