TD-Suite: All Batteries Included Framework for Technical Debt Classification

TL;DR

TD-Suite presents an end-to-end, transformer-based framework for automated technical debt classification from textual artifacts such as issue reports. It combines a modular data pipeline, robust training features (cross-validation, early stopping, class weighting) and an inference engine with Gradio UI and Docker deployment to support binary debt detection and debt-type classification. Empirical results show state-of-the-art performance with DeBERTaV3 and favorable resource-efficiency trade-offs via DistilRoBERTa, along with a proposed two-stage ensemble approach to improve type discrimination while mitigating multi-class weaknesses. The work emphasizes sustainability by tracking carbon emissions during training and inference, and it provides open-source replication assets to facilitate adoption and benchmarking in industry and research. Overall, TD-Suite offers a practical, scalable platform for automated TD analysis that integrates data handling, model training, evaluation, deployment, and sustainability considerations into software development workflows.

Abstract

Recognizing that technical debt is a persistent and significant challenge requiring sophisticated management tools, TD-Suite offers a comprehensive software framework specifically engineered to automate the complex task of its classification within software projects. It leverages the advanced natural language understanding of state-of-the-art transformer models to analyze textual artifacts, such as developer discussions in issue reports, where subtle indicators of debt often lie hidden. TD-Suite provides a seamless end-to-end pipeline, managing everything from initial data ingestion and rigorous preprocessing to model training, thorough evaluation, and final inference. This allows it to support both straightforward binary classification (debt or no debt) and more valuable, identifying specific categories like code, design, or documentation debt, thus enabling more targeted management strategies. To ensure the generated models are robust and perform reliably on real-world, often imbalanced, datasets, TD-Suite incorporates critical training methodologies: k-fold cross-validation assesses generalization capability, early stopping mechanisms prevent overfitting to the training data, and class weighting strategies effectively address skewed data distributions. Beyond core functionality, and acknowledging the growing importance of sustainability, the framework integrates tracking and reporting of carbon emissions associated with the computationally intensive model training process. It also features a user-friendly Gradio web interface in a Docker container setup, simplifying model interaction, evaluation, and inference.

Figures (1)

• Figure 1: TD-Suite High level Architecture