How Relevant is Software Engineering Research as Perceived by Practitioners? – ESEC/FSE 2015

The number of software engineering research papers over the last few years has grown significantly. An important question here is: how relevant is software engineering research to practitioners in the field? To address this question, we conducted a survey at Microsoft where we invited 3,000 industry practitioners to rate the relevance of research ideas contained in 571 papers ICSE and FSE papers that were published over a five year period. We received 17,913 ratings by 512 practitioners who labelled ideas as essential, worthwhile, unimportant, or unwise. The results from the survey suggest that practitioners are positive towards studies done by the software engineering research community: 71% of all ratings were essential or worthwhile. We found no correlation between the citation counts and the relevance scores of the papers. Through a qualitative analysis of free text responses, we identify several reasons why practitioners considered certain research ideas to be unwise. The survey approach is this paper is very lightweight, participants spent only 22.5 minutes to respond to the survey. At the same, the results can provide useful insights to conference organizers, authors, and the participating practitioners.

[click for more details…]

Quantifying Developers’ Adoption of Security Tools – ESEC/FSE 2015

Security tools could help developers find critical vulnerabilities, yet such tools remain underused. We surveyed developers from 14 companies and 5 mailing lists about their reasons for using and not using security tools. The resulting thirty-nine predictors of security tool use provide both expected and unexpected insights. As we expected, developers who perceive security to be important are more likely to use security tools than those who do not. However, that was not the strongest predictor of security tool use, it was instead developers’ ability to observe their peers using security tools.

[click for more details…]

Software Analytics for Digital Games – GAS 2015 Keynote

I presented the following keynote at the Fourth International Workshop on Games and Software Engineering (GAS 2015)

Software and its development generates an inordinate amount of data. Development activities such as check-ins, work items, bug reports, code reviews, and test executions are recorded in software repositories. User interactions that reflect how customers experience software are recorded in telemetry data, run-time traces, and log files and helps to track application and feature usage and expose performance and reliability. Software analytics takes this data and turns it into actionable insight to better inform decisions related to software.

In this talk, I will summarize our efforts in the area of software analytics with a special focus on digital games. I will present several examples of games studies, that we have worked on at Microsoft Research such as how players are engaged in Project Gotham Racing, how skill develops over time in Halo Reach and Forza Motorsports, and the initial experience of game play. I will also point out important differences between games development and traditional software development. The work presented in this talk has been done by Nachi Nagappan, myself, and many others who have visited our group over the past years.

When do Changes Induce Fixes? – Ten Years Later – MSR 2015

Here are the slides for a short talk that Andreas Zeller and I presented at MSR 2015 about the MSR 2005 paper When Do Changes Induce Fixes?, which won the Ten Year Most Influential Paper Award.

See you at the #icse15 SEIP track! Don’t miss it. It’s going to be awesome.

ICSE 2015 SEIP Track in Florence, Italy

If you are attending ICSE in Florence, please come to the SEIP sessions. This year the SEIP track was chaired by Antonino Sabetta and myself. We have a fantastic program for you!

The keynote presentation on “The Future of Software Engineering” by Grady Booch from IBM Research.

The presentation of “An Industrial Case Study on the Automated Detection of Performance Regressions in Heterogeneous Environments” (preprint), which received the the IEEE Software Best Paper Award for the Software Engineering in Practice track. Congratulations to King Chun Foo, Zhen Ming (Jack) Jiang, Bram Adams, Ahmed E. Hassan, Ying Zou, and Parminder Flora.

A total of 25 technical presentations by authors from many companies: ABB Inc., Airbus Group Innovations, BlackBerry, Cactuslab, Cisco Systems, Critiware, Daimler AG, Econocom-Osiatis, Google, IBM, LG Electronics, Microsoft Corp., NASA Goddard Space Flight Center, SafeRiver, Selex ES, SoftServe Inc., Software Improvement Group, Volvo Group Trucks Technology, and Wotan Engineering GmbH. (Note that two presentations were submitted as talk proposals this year.)

Thanks to the PC for selecting the fantastic program and the authors for submitting many great papers. We received 102 papers submissions and selected only 23 (a 22.5% acceptance rate) for the SEIP program.

See you in Florence.

Software Engineering Mix 2015

Software Engineering Mix 2015

If you are coming for the Microsoft Research Faculty Summit (or happen to live near Seattle), please sign up for the Software Engineering Mix, that I’m co-organizing with Andrew Begel. It’s going to be awesome!

Software Engineering Mix provides a forum for our colleagues from academia to interact directly with Microsoft engineers. The program will feature talks from academics: highlights of published research that is highly relevant for Microsoft and blue sky talks summarizing emerging research areas. In addition, practitioners will give presentations about theoretical and pragmatic engineering challenges they face, perhaps soliciting help from academia. A coffee round table setting will be used to facilitate discussions. This session builds on the success of SEIF Days, which provided a discussion forum about the future of software engineering.

Software Engineering Mix is part of the Microsoft Research Faculty Summit 2015.

SCAM 2015: International Working Conference on Source Code Analysis & Manipulation – Bremen, Germany

Please submit to the International Working Conference on Source Code Analysis & Manipulation (SCAM 2015), which will be held September 27-28, 2015 in Bremen, Germany (co-located with ICSME 2015). The submission deadline for the SCAM conference is June 26 (abstracts June 19; as always, please check the webpage for any extensions). I’m a member of the Program Committee for the technical research track.

The aim of the International Working Conference on Source Code Analysis & Manipulation (SCAM) is to bring together researchers and practitioners working on theory, techniques and applications which concern analysis and/or manipulation of the source code of computer systems. While much attention in the wider software engineering community is properly directed towards other aspects of systems development and evolution, such as specification, design and requirements engineering, it is the source code that contains the only precise description of the behavior of the system. The analysis and manipulation of source code thus remains a pressing concern.