It Will Never Work in Theory

Short summaries of recent results in empirical software engineering research

2021-12-13: Are Refactorings to Blame? An Empirical Study of Refactorings in Merge Conflicts
Keywords: Benchmarking, Code Smells
Reviewed by: Nazmus Sakeef

Merge conflicts are one of the biggest difficulties developers face when using separate development branches. Many developers believe that code refactoring makes the situation worse, but are the refactorings solely to blame? How big of a difference do they make when it comes to merge conflicts? Using data from roughly 3000 open-source Java software repositories, Mahmoudi2019 examined the association between merge conflicts and 15 prominent refactoring patterns, and at how difficult it is to settle refactoring-related disagreements. They used a tool called RefactoringMiner to detect refactorings and investigated the relationship between refactorings and merge conflicts; they also developed and presented...

2021-12-02: Deep Code Search
Keywords: Code Search, Deep Learning
Reviewed by: Hamza Alvi

Kim is a developer, and while implementing a new feature, they remember that they wrote something similar in another project. They open that project and starts searching for the code; It takes some time, but finding that code makes their current task easier. They wonder if the time spent searching was worth it, because they might have taken the same amount of time to reimplement the feature. To help developers with tasks like this, Gu2018 proposes a tool called DeepCS that takes natural language queries and searches for relevant code in a large codebase. The CODEnn model that DeepCS uses...

2021-11-29: Investigating Next Steps in Static API-Misuse Detection
Keywords: Code Smells
Reviewed by: Maliha Sultana

Developers often use APIs to access data or web applications, but misuse of APIs can lead to data loss or software crashes. Although several API misuse detectors exist, most of them suffer from low precision and recall. To address this, Sven2019 proposes MUDETECT, an API misuse detector which builds on the strengths and addresses many of the weaknesses of previous detectors. It has 2X higher recall than previous detectors and 2.5X higher precision; moreover, it can work in cross-project settings and mines patterns across projects rather than from only the target project. MUDETECT begins by encoding API usages as API-Usage...

2021-11-26: An Empirical Study of Metric-based Comparisons of Software Libraries
Keywords: Development Practices, Metrics
Reviewed by: Samridhi Vaid

Selecting the best software libraries for a task is a crucial step in programming, but often difficult and confusing because there are gazillions of Q&A websites with conflicting or out-of-date advice. These websites do not provide comprehensive descriptions of libraries' features and usually do not provide comparative data on different libraries. Lopez2018 tackled this problem by implementing and assessing a metric-based comparison of software libraries. They calculated nine software metrics for 34 open-source Java libraries belonging to ten popular domains, such as testing, databases, and cryptography. They then surveyed 61 participants of varied backgrounds to determine how useful the metric-based...

2021-11-23: The Seven Sins: Security Smells in Infrastructure as Code Scripts
Keywords: Devops, Infrastucture as code, Security Smell
Reviewed by: Brij Patel

Developers use infrastructure as code (IaC) scripts to provision servers and development environments. As with any code, they may unintentionally create security holes while writing these scripts. The purpose of Rahman2019 is to help them avoid doing this by analyzing real IaC scripts to identify security smells and developing a static analysis tool to identify occurrences. The authors started by collecting 1726 Puppet scripts from 74 repositories belonging to Mozilla, OpenStack, and Wikimedia Commons. By performing a qualitative analysis on these scripts, the authors identified seven smells: admin by default empty password hard-coded secret invalid IP address binding suspicious comment...

2021-11-18: Googling for Software Development: What Developers Search For and What They Find
Keywords: Data Science, Development Practices
Reviewed by: Kushankur Ghosh

Searching online serves as a base to solve problems and has become an inseparable part of our regular life. Developers in the software industry also rely on extracting relevant information from Google and other search engines to maximize their efficiency. A comprehensive illustration of the web-search queries from the perspective of software developers is provided in Hora2021a. By analyzing 1.3M search queries from developers, the authors found that: Almost half of the queries (54.5%) from the developer’s end refer to programming languages and software technologies. Queries are short and are mostly focused on searching APIs and for code examples that...

2021-11-18: SOAR: A Synthesis Approach for Data Science API Refactoring
Keywords: Automation, Data Science, Maintenance
Reviewed by: Ravika Nagpal

Everything that has been constructed eventually needs maintenance: homes, parks, offices---and code. Refactoring is one way programmers do this. Though the idea is simple, it can quickly become a programmer's nightmare, as manual refactoring is tedious and error-prone. Ni2021 introduce an automated refactoring technique called SOAR that combines natural language processing with program synthesis to automatically migrate and refactor between different versions of APIs. It begins by constructing an API matching model based on available documentation for the source and target libraries to find potential replacement calls for each API call in the source program. It then employs program synthesis...

2021-11-16: On Using Stack Overflow Comment-Edit Pairs to Recommend Code Maintenance Changes
Keywords: Maintenance, Stack Overflow, Usability
Reviewed by: Tajkia Rahman Toma

Have you ever developed any software without the help of Stack Overflow? In just 13 years, Stack Overflow has become an essential platform to get programming help. It has also become a source of data for researchers to solve many software engineering problems. However, Tang2021 are the first to evaluate whether Stack Overflow comments and edits can help solve the software maintenance problems. Their main goal was to find out how useful Stack Overflow comment-edit pairs are in making tools for code maintenance tasks like program repair and code recommender systems. The authors used SOTorrent to automatically mine edits of...

2021-11-16: Software History under the Lens: A Study on Why and How Developers Examine It
Keywords: Software Evolution, Tools, Version Control
Reviewed by: Rohan Verma

If you have ever wondered why and how developers analyze software history, or if you are a tool builder who is unsure of which features to include in your software history tool, then Codoban2015 may have some answers for you. They interviewed 14 experienced developers to examine developers' motivation, needs, and strategy for software history analysis, as well as the challenges they face during analysis. The authors then surveyed 217 people to extend these interviews' findings. Codoban2015 found that reasons like debugging, reverse engineering, understanding the ongoing work, keeping up with changes, or analyzing the impact of a change can...

2021-11-08: BugSwarm: Mining and Continuously Growing a Dataset of Reproducible Failures and Fixes
Keywords: Benchmarking, Bug Reports, DevOps
Reviewed by: Alexandru Ianta

A dataset! A dataset! My kingdom for a dataset! The sentiment behind the King Richard's famous cries is perhaps shared, in a different context, by contemporary researchers who have their sights set on developing bug detection or automatic repair systems. Luckily Tomassi2019 have put shovels in the ground and made impressive headway towards a hyper-scale, real-world, current, and reproducible dataset of bugs and their corresponding fixes. The progress takes the form of BugSwarm, a continuous integration (CI) harvesting toolkit that aims to take failed CI pipelines and turn them into data samples. Before BugSwarm, the datasets that were available for...