It Will Never Work in Theory

Jun 17, 2016
Keywords: Requests
Have you read any empirical software engineering research papers recently that you think a wider audience would enjoy? If so, please send us pointers: we'd be happy to feature them. (But please note that we only discuss work that is openly available—nothing paywalled, please.)
Jun 15, 2016
An Interview with Andreas Stefik
Keywords: Programming Languages
Functional Geekery's interview with Andreas Stefik is a great summary of what we actually know about the usability of programming languages—it's worth listening to the whole thing, especially the detailed discussion of studies of statically and dynamically typed languages.
Jun 13, 2016
Polymorphism in Python
Keywords: Programming Languages
Beatrice Åkerblom and Tobias Wrigstad: "Measuring Polymorphism in Python Programs". SPLASH'15, October 2015, In a break from our usual practice of quoting abstracts, here are this paper's conclusions: Our results show that while Python’s dynamic typing allows unbounded polymorphism, Python programs are predominantly monomorphic, that is, variables only hold values of a single type. This is true for program start-up and normal runtime, in library code and in program-specific code. Nevertheless, most programs have a few places which are megamorphic, meaning that variables in those places contain values of many different types at different times or in different contexts....
Jun 12, 2016
Frequency Distribution of Error Message
Keywords: Programming Languages
David Pritchard: "Frequency Distribution of Error Messages". PLATEAU'15, October 2015, Which programming error messages are the most common? We investigate this question, motivated by writing error explanations for novices. We consider large data sets in Python and Java that include both syntax and run-time errors. In both data sets, after grouping essentially identical messages, the error message frequencies empirically resemble Zipf-Mandelbrot distributions. We use a maximum-likelihood approach to fit the distribution parameters. This gives one possible way to contrast languages or compilers quantitatively. Based on a large corpus of error messages, the 5 most common errors in Python programs...
Jun 12, 2016
Parallelism in Open Source Projects
Keywords: Parallelism
Marc Kiefer, Daniel Warzel, and Walter Tichy: "An Empirical Study on Parallelism in Modern Open-Source Projects". SEPS'15, October 2015, We present an empirical study of 135 parallel open-source projects in Java, C# and C++ ranging from small (<1000 lines of code) to very large (>2M lines of code) codebases. We examine the projects to find out how language features, synchronization mechanisms, parallel data structures and libraries are used by developers to express parallelism. We also determine which common parallel patterns are used and how the implemented solutions compare to typical textbook advice. The results show that similar parallel constructs...
Jun 9, 2016
Novice Programming Mistakes
Keywords: Novices
Amjad Altadmri and Neil C. C. Brown: "37 Million Compilations: Investigating Novice Programming Mistakes in Large-Scale Student Data". SIGCSE'15, March 2015,, Educators often form opinions on which programming mistakes novices make most often – for example, in Java: "they always confuse equality with assignment", or "they always call methods with the wrong types". These opinions are generally based solely on personal experience. We report a study to determine if programming educators form a consensus about which Java programming mistakes are the most common. We used the Blackbox data set to check whether the educators' opinions matched data from...
Jun 9, 2016
Too Many Knobs
Keywords: Configuration
Tianyin Xu, Long Jin, Xuepeng Fan, Yuanyuan Zhou, Shankar Pasupathy, and Rukma Talwadker: "Hey, You Have Given Me Too Many Knobs! Understanding and Dealing with Over-Designed Configuration in System Software". ESEC/FSE'15, August 2015,, This paper makes a first step in understanding a fundamental question of configuration design: "do users really need so many knobs?" To provide the quantitatively answer, we study the configuration settings of real-world users, including thousands of customers of a commercial storage system (Storage-A), and hundreds of users of two widely-used open-source system software projects. Our study reveals a series of interesting findings to motivate...
Jun 9, 2016
A Software Engineering Homunculus
Keywords: Miscellaneous
A cortical homunculus is a graphical representation showing how much of the brain is devoted to different parts of the body: It would be fascinating to see a similar diagram showing how much software engineering research effort is devoted to which parts of the things software developers actually do. I strongly suspect, for example, that the ratio of research to practice is much greater than unity for software product families, while there is much (much) less research into installation and package management than there is practical wrangling. If someone has a grad student looking for a project, I'd be happy...
Jun 9, 2016
Perceived Relevance
Keywords: Relevance
David Log, Nachiappan Nagappan, and Thomas Zimmermann: "How Practitioners Perceive the Relevance of Software Engineering Research". ESEC/FSE'15, August 2015, The number of software engineering research papers over the last few years has grown significantly. An important question here is: how relevant is software engineering research to practitioners in the field? To address this question, we conducted a survey at Microsoft where we invited 3,000 industry practitioners to rate the relevance of research ideas contained in 571 ICSE, ESEC/FSE and FSE papers that were published over a five year period. We received 17,913 ratings by 512 practitioners who labelled ideas...
Jun 9, 2016
Hidden Truths in Dead Software Paths
Keywords: Program Analysis
Michael Eichberg, Ben Hermann, Mira Mezini, and Leonid Glanz: "Hidden Truths in Dead Software Paths". ESEC/FSE'15, August 2015, Approaches and techniques for statically finding a multitude of issues in source code have been developed in the past. A core property of these approaches is that they are usually targeted towards finding only a very specific kind of issue and that the effort to develop such an analysis is significant. This strictly limits the number of kinds of issues that can be detected. In this paper, we discuss a generic approach based on the detection of infeasible paths in...
Jun 9, 2016
Goto in C
Keywords: Programming Languages
Meiyappan Nagappan, Romain Robbes, Yasutaka Kamei, Eric Tante, Shane McIntos4, Audris Mocku5,and Ahmed E. Hassa: "An Empirical Study of Goto in C Code from GitHub Repositories". ESEC/FSE'15, August 2015, It is nearly 50 years since Dijkstra argued that goto obscures the flow of control in program execution and urged programmers to abandon the goto statement. While past research has shown that goto is still in use, little is known about whether goto is used in the unrestricted manner that Dijkstra feared, and if it is 'harmful' enough to be a part of a post-release bug. We, therefore, conduct a...
Jun 8, 2016
When, How, and Why Developers (Do Not) Test in Their IDEs
Keywords: Testing
Moritz Beller, Georgios Gousios, Annibale Pannichella, and Andy Zaidman: "When, How, and Why Developers (Do Not) Test in Their IDEs". ESEC/FSE'15, August 2015,, We report on the surprising results of a large-scale field study with 416 software engineers whose development activity we closely monitored over the course of five months, resulting in over 13 years of recorded work time in their integrated development environments (IDEs). Our findings question several commonly shared assumptions and beliefs about testing and might be contributing factors to the observed bug proneness of software in practice: the majority of developers in our study does...
Jun 7, 2016
Pull-Based Development
Keywords: Collaboration
Georgios Gousios, Margaret-Anne Storey, and Alberto Bacchelli: "Work Practices and Challenges in Pull-Based Development: The Contributor's Perspective". ICSE, May 2016,, We conducted a survey with 645 top contributors to active OSS projects using the pull-based model on GitHub, the prevalent social coding site. We also analyzed traces extracted from corresponding GitHub repositories. Our research shows that: contributors have a strong interest in maintaining awareness of project status to get inspiration and avoid duplicating work, but they do not actively propagate information; communication within pull requests is reportedly limited to low-level concerns and contributors often use communication channels external...
Jun 7, 2016
Code Ownership Revisited
Keywords: Code Ownership
Patanamon Thongtanunam, , Shane McIntosh, , Ahmed E. Hassan, and , Hajimu Iida: "Revisiting Code Ownership and its Relationship with Software Quality in the Scope of Modern Code Review". ICSE, May 2016,, Code ownership establishes a chain of responsibility for modules in large software systems. Although prior work uncovers a link between code ownership heuristics and software quality, these heuristics rely solely on the authorship of code changes. In addition to authoring code changes, developers also make important contributions to a module by reviewing code changes. Indeed, recent work shows that reviewers are highly active in modern code...
Jun 5, 2016
Keywords: Debugging
Muhammad Ali Gulzar, Matteo Interlandi, Seunghyun Yoo, Sai Deep Tetali, Tyson Condie, Todd Millstein, and Miryung Kim: "BigDebug: Debugging Primitives for Interactive Big Data Processing in Spark". ICSE'16, May 2016,, Developers use cloud computing platforms to process a large quantity of data in parallel when developing big data analytics. Debugging the massive parallel computations that run in today's data-centers is time consuming and error-prone. To address this challenge, we design a set of interactive, real-time debugging primitives for big data processing in Apache Spark, the next generation data-intensive scalable cloud computing platform. This requires rethinking the notion of...
Apr 26, 2016
Maintaining Java ORM Code
Keywords: Maintenance
Tse-Hsun Chen, Weiyi Shang, Jinqiu Yang, Ahmed E. Hassan, and Michael W. Godfrey: "An Empirical Study on the Practice of Maintaining Object-Relational Mapping Code in Java Systems". MSR'16, May 2016, From the abstract: Despite the advantages of using ORM [object-relational mapping] frameworks, we observe several difficulties in maintaining ORM code (i.e., code that makes use of ORM frameworks) when cooperating with our industrial partner. After conducting studies on other open source systems, we find that such difficulties are common in other Java systems. Our study finds that i) ORM cannot completely encapsulate database accesses in objects or abstract the...
Apr 26, 2016
Java Exception Handling
Keywords: Programming Languages
Suman Nakshatri, Maithri Hegde, and Sahithi Thandra: "Analysis of Exception Handling Patterns in Java Projects: An Empirical Study". MSR'16, May 2016,, Exception handling is a powerful tool provided by many programming languages to help developers deal with unforeseen conditions. Java is one of the few programming languages to enforce an additional compilation check on certain subclasses of the Exception class through checked exceptions. As part of this study, empirical data was extracted from software projects developed in Java. The intent is to explore how developers respond to checked exceptions and identify common patterns used by them to deal...
Apr 26, 2016
Game Postmortems
Keywords: Project Management
Michael Washburn Jr., Pavithra Sathiyanarayanan, Meiyappan Nagappan, Thomas Zimmermann, and Christian Bird: "'What Went Right and What Went Wrong': An Analysis of 155 Postmortems from Game Development". ICSE'16, May 2016,, In game development, software teams often conduct postmortems to reflect on what went well and what went wrong in a project. The postmortems are shared publicly on gaming sites or at developer conferences. In this paper, we present an analysis of 155 postmortems published on the gaming site We identify characteristics of game development, link the characteristics to positive and negative experiences in the postmortems and distill...
Apr 26, 2016
Finding Security Bugs
Keywords: Security
Joseph P. Near and Daniel Jackson: "Finding Security Bugs in Web Applications Using a Catalog of Access Control Patterns". ICSE'16, May 2016,, We propose a specification-free technique for finding missing security checks in web applications using a catalog of access control patterns in which each pattern models a common access control use case. Our implementation, SPACE, checks that every data exposure allowed by an application's code matches an allowed exposure from a security pattern in our catalog. The only user-provided input is a mapping from application types to the types of the catalog; the rest of the process...
Apr 26, 2016
Extremely Angry
Keywords: Relevance
From this thread by Mike Hoye, the Engineering Community Manager for Firefox at Mozilla: Extremely angry with the state of academic CS research right now. We've got a bunch of research papers that have come out of Mozilla's willingness to share a lot of our data. Bugzilla, lots of other stuff. We pay attention to that research. We read it carefully, and our operational decision-making is informed by those results. But we live and work in an evolving world, so the other thing we do as a matter of course is re-check our data to confirm our assumptions. What I...
Comments powered by Disqus