Bug Destiny Prediction in Large Open-Source Software Repositories through Sentiment Analysis and BERT Topic Modeling
Sophie C. Pope, Andrew Barovic, Armin Moin
TL;DR
This paper tackles predicting bug destiny and time-to-resolution outcomes using pre-resolution textual data from open-source bug repositories, focusing on Eclipse Bugzilla. It combines sentiment analysis with BERTopic-derived topics and employs CNN and MLP architectures, including data balancing strategies, to predict time-to-resolution, time-to-fix, and resolution labels. The findings show sentiment and topic features can improve certain predictions, particularly for whether a bug will be fixed, while balancing enhances practical applicability at the cost of accuracy in many cases; numeric time predictions remain challenging and destiny predictions achieve moderate accuracy. The work contributes an open-source prototype and demonstrates that before-resolution textual signals can meaningfully inform bug triage decisions in large-scale OSS projects.
Abstract
This study explores a novel approach to predicting key bug-related outcomes, including the time to resolution, time to fix, and ultimate status of a bug, using data from the Bugzilla Eclipse Project. Specifically, we leverage features available before a bug is resolved to enhance predictive accuracy. Our methodology incorporates sentiment analysis to derive both an emotionality score and a sentiment classification (positive or negative). Additionally, we integrate the bug's priority level and its topic, extracted using a BERTopic model, as features for a Convolutional Neural Network (CNN) and a Multilayer Perceptron (MLP). Our findings indicate that the combination of BERTopic and sentiment analysis can improve certain model performance metrics. Furthermore, we observe that balancing model inputs enhances practical applicability, albeit at the cost of a significant reduction in accuracy in most cases. To address our primary objectives, predicting time-to-resolution, time-to-fix, and bug destiny, we employ both binary classification and exact time value predictions, allowing for a comparative evaluation of their predictive effectiveness. Results demonstrate that sentiment analysis serves as a valuable predictor of a bug's eventual outcome, particularly in determining whether it will be fixed. However, its utility is less pronounced when classifying bugs into more complex or unconventional outcome categories.