When and How to Make Breaking Changes

Reviewed by Greg Wilson / 2022-03-25
Keywords: Software Evolution

Why do developers break backward compatibility? When do they do this, and do different communities make decisions the same way or do values and practices differ from one group to the next? Bogart2021 performed two studies to answer these questions. In the first, they conducted a multiple case study of the Eclipse, R/CRAN, and Node.js/npm ecosystems. They found substantial differences between them, and showed that:

…these differences arise from different values in each community and are reinforced through peer pressure, policies, and tooling. For example, long-term stability is a central value of the Eclipse community, achieved by their "prime directive" practice of never permitting breaking changes. This practice imposes costs on upstream developers, who may accept substantial opportunity costs and technical debt to avoid breaking client code. In contrast, the Node.js/npm community values ease and simplicity for upstream developers and has a technical infrastructure in which breaking changes are accepted, but signaled clearly through version numbering.

Their second study looked at a larger set of 18 ecosystems using data mining of software repositories, document analysis, and a large-scale survey. This study found that values and practices are cohesive within any particular ecosystem, but diverse across different ecosystems, and that even when ecosystems share similar values, they often achieve them in different ways (or fail to achieve it at all). Someone joining a new development community may therefore find that learning the technology is only part of the challenge: figuring out what their new peers expect and why can be just as hard.

At 56 pages, this is one of the longest papers we've reviewed, but it repays careful reading. The best part, though, is that the authors have made their data available for reanalysis (https://doi.org/10.1184/R1/5108716.v1) and have built a website to share their findings (http://breakingapis.org/).

Bogart2021 Chris Bogart, Christian Kästner, James Herbsleb, and Ferdian Thung: When and how to make breaking changes. ACM Trans. Software Engineering and Methodology, 30(4), 2021, doi:10.1145/3447245.

Open source software projects often rely on package management systems that help projects discover, incorporate, and maintain dependencies on other packages, maintained by other people. Such systems save a great deal of effort over ad hoc ways of advertising, packaging, and transmitting useful libraries, but coordination among project teams is still needed when one package makes a breaking change affecting other packages. Ecosystems differ in their approaches to breaking changes, and there is no general theory to explain the relationships between features, behavioral norms, ecosystem outcomes, and motivating values. We address this through two empirical studies. In an interview case study, we contrast Eclipse, NPM, and CRAN, demonstrating that these different norms for coordination of breaking changes shift the costs of using and maintaining the software among stakeholders, appropriate to each ecosystem's mission. In a second study, we combine a survey, repository mining, and document analysis to broaden and systematize these observations across 18 ecosystems. We find that all ecosystems share values such as stability and compatibility, but differ in other values. Ecosystems' practices often support their espoused values, but in surprisingly diverse ways. The data provides counterevidence against easy generalizations about why ecosystem communities do what they do.