Automatic Extraction of Bug Localization Benchmarks from History – Technical Report

by Valentin Dallmeier, Thomas Zimmermann

Researchers have proposed a number of tools for automatic bug localization. Given a program and a description of the failure, such tools pinpoint a set of statements that are most likely to contain the bug. Evaluating bug localization tools is a difficult task because existing benchmarks are limited in size of subjects and number of bugs. In this paper we present IBUGS, an approach that automatically extracts benchmarks for bug localization from the history of a project. For ASPECTJ, we extracted 369 bugs, 223 out of these had associated test cases (useful to test dynamic tools). We demonstrate the relevance of our dataset for both static and dynamic bug localization tools with case studies on FINDBUGS and AMPLE.

Download as PDF.
See also:


Valentin Dallmeier, Thomas Zimmermann. Automatic Extraction of Bug Localization Benchmarks from History. Technical Report, June 2007. Accepted at ASE 2007. Please cite the conference paper.

BibTeX Entry

    title = "Automatic Extraction of Bug Localization Benchmarks from History",
    author = "Valentin Dallmeier and Thomas Zimmermann",
    year = "2007",
    month = "June",
    institution = "Universität des Saarlandes, Saarbrücken, Germany",