Beyond the Code Itself: How Programmers Really Look at Pull Requests

If I ever teach another full-semester course on software engineering, I'm going to devote at least one lecture to cognitive bias. None of us make decisions as rationally as we'd like to believe, but knowing how our thinking can be skewed is the first step toward thinking more clearly.

Ford2019 used eye-tracking to find out what programmers actually look at when evaluating pull requests on GitHub, and compared those findings with what programmers said they did. Their conclusion? "…after the code snippet, the second place programmers spent their time fixating is on supplemental technical signals, such as previous contributions and popular repositories. Diverging from what participants reported themselves, we also found that programmers fixated on social signals more than recalled."

This study also found that programmers use different strategies to protect their identity on GitHub than the use on Facebook. Those strategies included:

• Stay aware of image presented and how they will be perceived on each platform.
• Make the code stand alone, regardless of the name attached.
• When in doubt, use an anonymous name.
• Complete the online profile to be perceived as trustworthy.
• Create personal rules for sharing content based on the platform's primary function.

As a middle-aged straight white male with no significant physical or mental challenges who is economically secure and fluent in English, I can afford to pretend that "the code speaks for itself". I don't know how often I've made poor decisions because of that blindspot, but I hope teachers will incorporate insights like those in this paper into their courses so that the next generation of programmers will make better ones.