A Meta-analytical Comparison of Naive Bayes and Random Forest for Software Defect Prediction
Ch Muhammad Awais, Wei Gu, Gcinizwe Dlamini, Zamira Kholmatova, Giancarlo Succi
TL;DR
This paper investigates whether Naive Bayes (NB) and Random Forest (RF) differ in predicting software defects. It employs a two-stage approach—systematic literature review followed by meta-analysis—across five studies focusing on recall, precision, and F-measure, and computes standardized effect sizes using Hedge's correction within fixed and random effects frameworks. The results, synthesized via forest plots, show no statistically significant difference between NB and RF across the evaluated metrics, suggesting comparable utility in defect prediction. The work underscores the need for larger, more diverse datasets and consistent reporting to enable robust cross-study comparisons and more definitive conclusions about classifier performance in software engineering contexts.
Abstract
Is there a statistical difference between Naive Bayes and Random Forest in terms of recall, f-measure, and precision for predicting software defects? By utilizing systematic literature review and meta-analysis, we are answering this question. We conducted a systematic literature review by establishing criteria to search and choose papers, resulting in five studies. After that, using the meta-data and forest-plots of five chosen papers, we conducted a meta-analysis to compare the two models. The results have shown that there is no significant statistical evidence that Naive Bayes perform differently from Random Forest in terms of recall, f-measure, and precision.