A Theoretical and Empirical Taxonomy of Imbalance in Binary Classification
Rose Yvette Bandolo Essomba, Ernest Fokoué
TL;DR
The paper addresses how class imbalance interacts with high dimensionality and intrinsic separability to degrade binary classification. It develops a Bayes-risk framework parameterized by the triplet $(η,κ,Δ)$, deriving closed-form Bayes errors under a Gaussian model and revealing how the decision boundary shifts with $\,\log(η)$ and how the effective margin scales with $Δ\sqrt{κ}$. A principled regime taxonomy—Normal, Mild, Extreme, and Catastrophic—is introduced, with analytical predictions for minority recall, precision, F1, and PR-AUC that are validated empirically across parametric and non-parametric models. The triplet framework provides a model-agnostic, geometry-based lens for predicting deterioration under imbalance and motivates imbalance-aware algorithm design.
Abstract
Class imbalance significantly degrades classification performance, yet its effects are rarely analyzed from a unified theoretical perspective. We propose a principled framework based on three fundamental scales: the imbalance coefficient $η$, the sample--dimension ratio $κ$, and the intrinsic separability $Δ$. Starting from the Gaussian Bayes classifier, we derive closed-form Bayes errors and show how imbalance shifts the discriminant boundary, yielding a deterioration slope that predicts four regimes: Normal, Mild, Extreme, and Catastrophic. Using a balanced high-dimensional genomic dataset, we vary only $η$ while keeping $κ$ and $Δ$ fixed. Across parametric and non-parametric models, empirical degradation closely follows theoretical predictions: minority Recall collapses once $\log(η)$ exceeds $Δ\sqrtκ$, Precision increases asymmetrically, and F1-score and PR-AUC decline in line with the predicted regimes. These results show that the triplet $(η,κ,Δ)$ provides a model-agnostic, geometrically grounded explanation of imbalance-induced deterioration.