Local vs Global Models for Effort Estimation and Defect Prediction – ASE 2011

by Tim Menzies, Andrew Butcher, Andrian Marcus, Thomas Zimmermann, David Cok

Data miners can infer rules showing how to improve either (a) the effort estimates of a project or (b) the defect predictions of a software module. Such studies often exhibit conclusion instability regarding what is the most effective action for different projects or modules.

This instability can be explained by data heterogeneity. We show that effort and defect data contain many local regions with markedly different properties to the global space. In other words, what appears to be useful in a global context is often irrelevant for particular local contexts.

This result raises questions about the generality of conclusions from empirical SE. At the very least, SE researchers should test if their supposedly general conclusions are valid within subsets of their data. At the very most, empirical SE should become a search for local regions with similar properties (and conclusions should be constrained to just those regions).

Download as PDF.

Reference

Tim Menzies, Andrew Butcher, Andrian Marcus, Thomas Zimmermann, David Cok. Local vs Global Models for Effort Estimation and Defect Prediction. In Proceedings of the 26st IEEE/ACM International Conference on Automated Software Engineering (ASE 2011), Lawrence, Kansas, USA, November 2011.

BibTeX Entry

@inproceedings{menzies-ase-2011,
    title = "Local vs Global Models for Effort Estimation and Defect Prediction",
    author = "Tim Menzies and Andrew Butcher and Andrian Marcus and Thomas Zimmermann and David Cok",
    year = "2011",
    month = "November",
    booktitle = "Proceedings of the 26st IEEE/ACM International Conference on Automated Software Engineering",
    location = "Lawrence, Kansas, USA",
}