Retrospective: Data Mining Static Code Attributes to Learn Defect Predictors
Tim Menzies
TL;DR
The paper reflects on the 2007 study that used static code attributes to predict software defects and popularized reproducible SE via the PROMISE data repository. It traces the rise of PROMISE, its industry impact, and the subsequent shift toward data-quality concerns and newer collection practices. It highlights ongoing and future directions, including deep learning, interpretability, and semi-/self-supervised learning, while cautioning about data quality, over-reliance on old datasets, and the brittleness of some conclusions. Overall, it argues that open-science baselines remain valuable but require evolution to match modern data practices and evaluation standards.
Abstract
Industry can get any research it wants, just by publishing a baseline result along with the data and scripts need to reproduce that work. For instance, the paper ``Data Mining Static Code Attributes to Learn Defect Predictors'' presented such a baseline, using static code attributes from NASA projects. Those result were enthusiastically embraced by a software engineering research community, hungry for data. At its peak (2016) this paper was SE's most cited paper (per month). By 2018, twenty percent of leading TSE papers (according to Google Scholar Metrics), incorporated artifacts introduced and disseminated by this research. This brief note reflects on what we should remember, and what we should forget, from that paper.