It Will Never Work in Theory

Short summaries of recent results in empirical software engineering research

2021-11-26: An Empirical Study of Metric-based Comparisons of Software Libraries
Keywords: Development Practices, Metrics
Reviewed by: Samridhi Vaid

Selecting the best software libraries for a task is a crucial step in programming, but often difficult and confusing because there are gazillions of Q&A websites with conflicting or out-of-date advice. These websites do not provide comprehensive descriptions of libraries' features and usually do not provide comparative data on different libraries. Lopez2018 tackled this problem by implementing and assessing a metric-based comparison of software libraries. They calculated nine software metrics for 34 open-source Java libraries belonging to ten popular domains, such as testing, databases, and cryptography. They then surveyed 61 participants of varied backgrounds to determine how useful the metric-based...

2021-11-23: The Seven Sins: Security Smells in Infrastructure as Code Scripts
Keywords: Devops, Infrastucture as code, Security Smell
Reviewed by: Brij Patel

Developers use infrastructure as code (IaC) scripts to provision servers and development environments. As with any code, they may unintentionally create security holes while writing these scripts. The purpose of Rahman2019 is to help them avoid doing this by analyzing real IaC scripts to identify security smells and developing a static analysis tool to identify occurrences. The authors started by collecting 1726 Puppet scripts from 74 repositories belonging to Mozilla, OpenStack, and Wikimedia Commons. By performing a qualitative analysis on these scripts, the authors identified seven smells: admin by default empty password hard-coded secret invalid IP address binding suspicious comment...

2021-11-18: Googling for Software Development: What Developers Search For and What They Find
Keywords: Data Science, Development Practices
Reviewed by: Kushankur Ghosh

Searching online serves as a base to solve problems and has become an inseparable part of our regular life. Developers in the software industry also rely on extracting relevant information from Google and other search engines to maximize their efficiency. A comprehensive illustration of the web-search queries from the perspective of software developers is provided in Hora2021a. By analyzing 1.3M search queries from developers, the authors found that: Almost half of the queries (54.5%) from the developer’s end refer to programming languages and software technologies. Queries are short and are mostly focused on searching APIs and for code examples that...

2021-11-18: SOAR: A Synthesis Approach for Data Science API Refactoring
Keywords: Automation, Data Science, Maintenance
Reviewed by: Ravika Nagpal

Everything that has been constructed eventually needs maintenance: homes, parks, offices---and code. Refactoring is one way programmers do this. Though the idea is simple, it can quickly become a programmer's nightmare, as manual refactoring is tedious and error-prone. Ni2021 introduce an automated refactoring technique called SOAR that combines natural language processing with program synthesis to automatically migrate and refactor between different versions of APIs. It begins by constructing an API matching model based on available documentation for the source and target libraries to find potential replacement calls for each API call in the source program. It then employs program synthesis...

2021-11-16: On Using Stack Overflow Comment-Edit Pairs to Recommend Code Maintenance Changes
Keywords: Maintenance, Stack Overflow, Usability
Reviewed by: Tajkia Rahman Toma

Have you ever developed any software without the help of Stack Overflow? In just 13 years, Stack Overflow has become an essential platform to get programming help. It has also become a source of data for researchers to solve many software engineering problems. However, Tang2021 are the first to evaluate whether Stack Overflow comments and edits can help solve the software maintenance problems. Their main goal was to find out how useful Stack Overflow comment-edit pairs are in making tools for code maintenance tasks like program repair and code recommender systems. The authors used SOTorrent to automatically mine edits of...

2021-11-16: Software History under the Lens: A Study on Why and How Developers Examine It
Keywords: Software Evolution, Tools, Version Control
Reviewed by: Rohan Verma

If you have ever wondered why and how developers analyze software history, or if you are a tool builder who is unsure of which features to include in your software history tool, then Codoban2015 may have some answers for you. They interviewed 14 experienced developers to examine developers' motivation, needs, and strategy for software history analysis, as well as the challenges they face during analysis. The authors then surveyed 217 people to extend these interviews' findings. Codoban2015 found that reasons like debugging, reverse engineering, understanding the ongoing work, keeping up with changes, or analyzing the impact of a change can...

2021-11-08: BugSwarm: Mining and Continuously Growing a Dataset of Reproducible Failures and Fixes
Keywords: Benchmarking, Bug Reports, DevOps
Reviewed by: Alexandru Ianta

A dataset! A dataset! My kingdom for a dataset! The sentiment behind the King Richard's famous cries is perhaps shared, in a different context, by contemporary researchers who have their sights set on developing bug detection or automatic repair systems. Luckily Tomassi2019 have put shovels in the ground and made impressive headway towards a hyper-scale, real-world, current, and reproducible dataset of bugs and their corresponding fixes. The progress takes the form of BugSwarm, a continuous integration (CI) harvesting toolkit that aims to take failed CI pipelines and turn them into data samples. Before BugSwarm, the datasets that were available for...

2021-10-31: Locating Faults with Program Slicing
Keywords: Debugging, Program Slicing
Reviewed by: Greg Wilson

Most programmers spend more time debugging than writing new code, but books and courses on debugging are few and far between, and testing tools to help people figure out if code is buggy or not are much more common than ones to help them figure out where the bugs are. Thankfully, there is a growing body of research on isolating and repairing bugs. One interesting approach is statistical fault localization: given a set of tests that pass and another set that fail, look at the correlation between statements that are executed by failing tests but not by passing tests in...

2021-10-30: App Store Effects on Software Engineering Practices
Keywords: App Store, Development Practices
Reviewed by: Greg Wilson

I once heard an athletics coach say, "If you change the ball, you have to change the game." What she meant was that you can't play football with a tennis ball or vice versa, and something similar is proving true in software engineering. Once enough people have reliable high-bandwidth connections, chat tools and package repositories become feasible, and development practices inevitably change in response. AlSubaihin2021 looks at the influence one innovation—app stores—are having on how developers work. The biggest impact is on requirements and design: most developers look in app stores to gauge the feasibility of their idea and for...

2021-10-26: Error Messages in Relational Database Systems
Keywords: Databases, Error Messages
Reviewed by: Eddie Antonio Santos

When I was first introduced to SQL in my undergraduate database course, we were trained on an Oracle SQL instance. Taipalus2021 provides evidence that I may have been just a bit happier had we used PostgreSQL instead. Taipalus2021 studied how choice of database management system (DBMS) affected the effectiveness and subjective experience of novices when correcting SQL syntax errors. Overall, novices reported that the error messages produced by PostgreSQL and Microsoft SQL Server were more useful than error messages from MySQL, meanwhile Oracle's error messages were usually deemed the least useful—perhaps because its messages generally do a poor job of...