Deep Learning and Data Augmentation for Detecting Self-Admitted Technical Debt
Edi Sutoyo, Paris Avgeriou, Andrea Capiluppi
TL;DR
The paper addresses the challenge of detecting Self-Admitted Technical Debt (SATD) and its types in four software artifacts under highly imbalanced data. It proposes a two-step pipeline: BiLSTM for SATD identification (SATD vs Not-SATD) and BERT for SATD-type categorization, augmented by AugGPT to balance minority classes. A balanced dataset (satdaug) is produced and evaluated against baselines, with results showing notable improvements in both identification and categorization, especially when augmentation is applied. The work also leverages KeyBERT to extract indicative keywords per artifact and type, contributing to explainability and practical search. The findings offer actionable guidance for researchers and practitioners on data augmentation, artifact-specific labeling, and keyword-based analysis to manage SATD more effectively in open-source software.
Abstract
Self-Admitted Technical Debt (SATD) refers to circumstances where developers use textual artifacts to explain why the existing implementation is not optimal. Past research in detecting SATD has focused on either identifying SATD (classifying SATD items as SATD or not) or categorizing SATD (labeling instances as SATD that pertain to requirement, design, code, test debt, etc.). However, the performance of these approaches remains suboptimal, particularly for specific types of SATD, such as test and requirement debt, primarily due to extremely imbalanced datasets. To address these challenges, we build on earlier research by utilizing BiLSTM architecture for the binary identification of SATD and BERT architecture for categorizing different types of SATD. Despite their effectiveness, both architectures struggle with imbalanced data. Therefore, we employ a large language model data augmentation strategy to mitigate this issue. Furthermore, we introduce a two-step approach to identify and categorize SATD across various datasets derived from different artifacts. Our contributions include providing a balanced dataset for future SATD researchers and demonstrating that our approach significantly improves SATD identification and categorization performance compared to baseline methods.