It Will Never Work in Theory

Short summaries of recent results in empirical software engineering research

2022-05-24: Marian Petre on Expert Thinking About Errors
Keywords: Lightning Talks
Reviewed by: Greg Wilson

Marian Petre: "How expert programmers think about errors." Marian is a Professor at the Open University in the United Kingdom. slides | transcript (English)

2022-05-24: Davide Fucci on Test-Driven Development
Keywords: Lightning Talks
Reviewed by: Greg Wilson

Davide Fucci: "The hidden costs and benefits of TDD." Davide is an Assistant Professor at the Blekinge Institute of Technology in Sweden. slides | transcript (English)

2022-05-24: Maurício Aniche on Testing and Code Coverage
Keywords: Lightning Talks
Reviewed by: Greg Wilson

Maurício Aniche: "How code coverage can be used and abused to guide testing." Maurício is Tech Academy Lead at Adyen and an Assistant Professor at Delft University of Technology in the Netherlands. slides | transcript (English)

2022-05-23: Mairieli Wessel on GitHub Actions
Keywords: Lightning Talks
Reviewed by: Greg Wilson

Mairieli Wessel: "How people really use GitHub Actions." Mairieli is a postdoctoral researcher at Delft University of Technology in the Netherlands. slides | transcript (English) | transcripción (Español)

2022-05-23: Matthias Galster on Waste in Software Engineering
Keywords: Lightning Talks
Reviewed by: Greg Wilson

Matthias Galster: "Value and waste in software engineering." Matthias is an Associate Professor at the University of Canterbury in New Zealand. slides | transcript (English) | transcripción (Español)

2022-05-20: Sebastian Baltes on Software Engineering in Papua New Guinea
Keywords: Lightning Talks
Reviewed by: Greg Wilson

Sebastian Baltes: "Challenges and opportunities for software engineering in Papua New Guinea." Sebastian is a Principal Expert for Empirical Software Engineering at SAP in Germany and an Adjunct Lecturer at the University of Adelaide in Australia. slides | transcript (English) | transcripción (Español)

2022-05-20: A Note on Communication
Keywords: Editorial
Reviewed by: Greg Wilson

I've been working full-time with experimental biologists for about seven months now, and the experience has reminded me of just how bad most software engineering researchers are at writing abstracts. An abstract isn't supposed to be like a movie trailer, designed to lure me into paying $13 to watch the whole thing; it's supposed to summarize the actual findings so that (a) I know a couple of things I didn't before, and (b) I can make an informed judgment about whether it's worth reading the whole thing. For example, the abstract of one bio paper I looked at recently was,...

2022-05-19: Felienne Hermans on Naming Things
Keywords: Lightning Talks
Reviewed by: Greg Wilson

Felienne Hermans: "How patterns in variable names can make code easier to read." Felienne is an Associate Professor at Leiden University in the Netherlands. slides | transcript (English) | traducción (Español)

2022-05-18: Alberto Bacchelli on Code Review
Keywords: Lightning Talks
Reviewed by: Greg Wilson

Alberto Bacchelli: "How code review works (and doesn't) in the real world." Alberto is an Associate Professor at the University of Zurich in Switzerland. slides | transcript (English) | transcripción (Español)

2022-05-14: May 2022 To Do
Keywords: Editorial
Reviewed by: Greg Wilson

We’re still working on the videos from our April 27 lightning talks, so I’ve taken some time to add 29 recent papers to our review queue: Abou Khalil and Zacchiroli: “The General Index of Software Engineering Papers” Ait et al: “An Empirical Study on the Survival Rate of GitHub Projects” AlOmar et al: “Code Review Practices for Refactoring Changes: An Empirical Study on OpenStack” Asare et al: “Is GitHub’s Copilot as Bad As Humans at Introducing Vulnerabilities in Code?” Bendrissou et al: “‘Synthesizing Input Grammars’: A Replication Study” Bogner and Merkel: “To Type or Not to Type? A Systematic Comparison...

2022-05-06: Not Quite Modern
Keywords: Book Review
Reviewed by: Greg Wilson

Despite its title, David Farley's new book Modern Software Engineering feels a bit old-fashioned. The author is best known for his advocacy of dev ops and continuous delivery, and his starting point is the following definition (pg 17): Engineering is the application of an empirical, scientific approach to finding efficient, economic solutions to practical problems. I strongly agree that our field needs more of this; I was therefore disappointed that Farley proceeds to leave out almost everything that empirical, scientific study of programs, programming, and programmers has uncovered in the last three decades. As just one example, consider test-driven development....

2022-04-30: That Went Well
Keywords: Lightning Talks
Reviewed by: Greg Wilson

Our first online event on April 27 is now over, and from all the feedback we’ve received it went very well. There were the inevitable technical hitches—some people couldn’t sign in to the discussion forum to ask questions, video conferencing locked up for a few minutes during one speaker’s presentation—but those were pretty minor when set against the engaging and informative presentations. Thank you again to all the presenters, to Taylor & Francis and Strange Loop for their sponsorship, and to Mozilla for providing technical support. We hope to have recordings of the presentations online in the next couple of...

2022-04-25: Extracting Rationale for Open Source Development Decisions
Keywords: Open Source
Reviewed by: Greg Wilson

I've never tried to measure how much of my time I spend staring at a screen full of code and wondering, "Why the hell is it doing that?" Even small teams with little turnover can struggle to keep track of the reasons for particular decisions, and newcomers frequently have to recapitulate their predecessors' journeys of discovery in order to figure out why alpha has to be re-initialized at that particular moment. Sharma2021 explores how well decision rationales can be extracted after the fact from Python's email archives. The authors combine term patterns such as Proposal-State-Reason, proximity-based heuristics, role-based heuristics, and...

2022-04-24: Role Clarity Deficiencies Can Wreck Agile Teams
Keywords: Agile Development
Reviewed by: Greg Wilson

I cannot do a better job of summarizing this paper on agile teams than the authors do themselves: Self-organization becomes very hard when there is incomplete understanding of the roles in the team; this happens suprisingly often; coach-facilitated reflection on roles can presumably help; achieving sufficiently complete understanding and acceptance is difficult even then. Once achieved, good role understanding and acceptance helps a team to be effective and work together well. The process of gaining role clarity is difficult and can be emotionally challenging. Technical skills can be less important than being self-reflective. Reading this paper has made me reflect...

2022-04-23: Moving From Closed to Open Source
Keywords: Open Source
Reviewed by: Greg Wilson

We often talk about commercial versus open source software, but in reality there has always been a continuum and a lot of migration in both directions: Many people have built successful businesses around open source software, and it is increasingly common for companies to take code they have developed in house and release it to (or in some cases, inflict it upon) the world. Kochhar2019 looks at how six internal Microsoft projects made that journey. Through interviews with stakeholders, surveys of developers working on the projects, and quantitative analysis of their contributors and communities, the authors explore the decisions that...

2022-04-21: A Cautionary Tale from the Decline of SourceForge
Keywords: Open Source
Reviewed by: Greg Wilson

There are thousands of books in print today about starting a business, but only a handful about ending one and moving on. Tamburri2020 is a deep dive into one particular (and not yet complete) ending: the decline of SourceForge, which was the first (and for many years the largest) open source software forge. Over the course of 22 pages, the authors look at how organizational siloing, the lone wolf effect, and other factors within projects interacted with things like its ill-conceived DevShare program, frequent changes of ownership, and the steady accumulation of technical debt in the site itself. While the...

2022-04-20: Assessing Exception Handling Testing Practices in Open-Source Libraries
Keywords: Open Source, Testing
Reviewed by: Greg Wilson

Eight years ago, Yuan2014 found that many of the failures in data-intensive systems were a result of people not testing the error handling, i.e., they didn't check that the alarms actually worked as designed. Subsequent research in several different contexts has confirmed this: programmers are less likely to test the "unhappy path" through their code, not least because of the difficulty of simulating things like timeouts, dropped network connections, and permission errors. Lima2021 is therefore welcome news. They examined the test suites of 27 well-known open-source Java libraries, and while they did find that instructions and branches in catch blocks...

2022-04-18: Andreas Zeller Summarizes ICSE 2022
Keywords: Conference
Reviewed by: Greg Wilson

Prof. Andreas Zeller has posted a short overview of current topics in software engineering research based on submissions to ICSE'22. From his thread and based on this data and this data: Machine learning is clearly a hot topic with 74 accepted papers, versus 47 for software testing (the scond most popular topic) 35 for program analysis, and 31 for evolution and maintenance. Only two papers each were accepted on ethics in software engineering, software architecture, green and sustainable technologies, and agile methods—which is still better than one each on model-driven engineering and software visualization. The majority of authors come from...

2022-04-18: Company Contributions to Open Source
Keywords: Open Source
Reviewed by: Greg Wilson

Somewhere in my basement I still have a copy of a memo I wrote in 1989 explaining to my boss why we could safely ignore all this "free software" nonsense that people were talking about on Usenet. Sure, a few zealots might make contributions, but programmers need to eat: they weren't just going to give their code away. I've rarely been so glad to be wrong, but I wasn't completely wrong. As the authors of Butler2019 point out, many contributions to major open source projects are made on behalf of companies, who employ core developers, make direct financial donations, and...

2022-04-14: Generating Unit Tests for Documentation
Keywords: Documentation
Reviewed by: Greg Wilson

This paper describes a tool called DScribe, which generates documentation and unit tests simultaneously from templates like this: (doc) If no argument is provided, the default value for the arguments of $method$ is $defVal$. (test) @Test public void test$method$_defaultArgument() { assertEquals( $method$() , $method$($defVal$) ); } It's a clever extension of things like Python's doctest, but is it actually useful? To find out, the authors studied the inconsistencies between source code, unit tests, and documentation in exceptions in the Apache Commons IO library. They found that: Among the sampled methods of Commons IO, only 15% of the specifications related to...