Finding the Sweet Spot: Optimal Data Augmentation Ratio for Imbalanced Credit Scoring Using ADASYN

TL;DR

Credit scoring models suffer from severe class imbalance, with defaults typically a minority. The study systematically compares data augmentation strategies (SMOTE, BorderlineSMOTE, ADASYN) across multiplication factors ($1\times$, $2\times$, $3\times$) using XGBoost on the Give Me Some Credit dataset, with bootstrap significance tests. It finds that ADASYN at the 1x multiplier yields the best performance ($AUC$=$0.6778$, $Gini$=$0.3557$) with a statistically significant improvement ($p=0.017$), while higher augmentation factors degrade performance, revealing a "sweet spot" near a 6.6:1 imbalance. The paper delivers practical guidance for practitioners to avoid full balancing, favor adaptive augmentation, and use a simple, reproducible evaluation framework, contributing a rigorous methodology for optimizing augmentation in imbalanced domains.

Abstract

Credit scoring models face a critical challenge: severe class imbalance, with default rates typically below 10%, which hampers model learning and predictive performance. While synthetic data augmentation techniques such as SMOTE and ADASYN have been proposed to address this issue, the optimal augmentation ratio remains unclear, with practitioners often defaulting to full balancing (1:1 ratio) without empirical justification. This study systematically evaluates 10 data augmentation scenarios using the Give Me Some Credit dataset (97,243 observations, 7% default rate), comparing SMOTE, BorderlineSMOTE, and ADASYN at different multiplication factors (1x, 2x, 3x). All models were trained using XGBoost and evaluated on a held-out test set of 29,173 real observations. Statistical significance was assessed using bootstrap testing with 1,000 iterations. Key findings reveal that ADASYN with 1x multiplication (doubling the minority class) achieved optimal performance with AUC of 0.6778 and Gini coefficient of 0.3557, representing statistically significant improvements of +0.77% and +3.00% respectively (p = 0.017, bootstrap test). Higher multiplication factors (2x and 3x) resulted in performance degradation, with 3x showing a -0.48% decrease in AUC, suggesting a "law of diminishing returns" for synthetic oversampling. The optimal class imbalance ratio was found to be 6.6:1 (majority:minority), contradicting the common practice of balancing to 1:1. This work provides the first empirical evidence of an optimal "sweet spot" for data augmentation in credit scoring, with practical guidelines for industry practitioners and researchers working with imbalanced datasets. While demonstrated on a single representative dataset, the methodology provides a reproducible framework for determining optimal augmentation ratios in other imbalanced domains.

Figures (3)

• Figure 1: Sweet Spot Phenomenon: Performance metrics (AUC and Gini) as a function of ADASYN augmentation multiplier. The inverted-U pattern demonstrates that 1$\times$ multiplier achieves optimal performance, while 2$\times$ shows diminishing returns and 3$\times$ leads to performance degradation below baseline. The gold star marks the optimal configuration.
• Figure 2: ROC curve comparison between Baseline and ADASYN 1$\times$. The green solid line (ADASYN 1$\times$) consistently outperforms the red dashed line (Baseline), achieving an AUC of 0.6778 compared to 0.6727, representing a +0.77% improvement. The area between the two curves represents the practical improvement in model performance.
• Figure 3: Lorenz curves comparing discrimination power between Baseline and ADASYN 1$\times$. The larger area between the ADASYN 1$\times$ curve and the equality line indicates superior ability to identify high-risk borrowers. The Gini coefficient improvement from 0.3453 to 0.3557 (+3.00%) demonstrates enhanced credit risk discrimination.