Position: Embracing Negative Results in Machine Learning
Florian Karl, Lukas Malte Kemeter, Gabriel Dax, Paulina Sierak
TL;DR
The paper argues that ML research overemphasizes predictive performance, which distorts incentives, wastes resources, and hampers reproducibility. It advocates embracing negative results (NMNR and EMNR) as a legitimate and valuable contribution to science, outlining concrete benefits such as improved replication, broader theoretical development, and better alignment with real-world impact. Through counterfactuals and practical measures, it proposes concrete actions—special venues, teaching updates, replication funding, and review-process reforms—to normalize publishing negative results. The authors emphasize that a balanced ecosystem, not a majority of negative-result papers, will yield healthier scientific output and longer-term progress. The work aims to spark discussion and drive cultural change in how ML research is evaluated and shared, with tangible steps to realize that shift.
Abstract
Publications proposing novel machine learning methods are often primarily rated by exhibited predictive performance on selected problems. In this position paper we argue that predictive performance alone is not a good indicator for the worth of a publication. Using it as such even fosters problems like inefficiencies of the machine learning research community as a whole and setting wrong incentives for researchers. We therefore put out a call for the publication of "negative" results, which can help alleviate some of these problems and improve the scientific output of the machine learning research community. To substantiate our position, we present the advantages of publishing negative results and provide concrete measures for the community to move towards a paradigm where their publication is normalized.