Online Bayesian Imbalanced Learning with Bregman-Calibrated Deep Networks
Zahir Alsulaimawi
TL;DR
This work introduces Online Bayesian Imbalanced Learning (OBIL), a framework that decouples likelihood-ratio estimation from deployment priors to enable real-time adaptation under distribution shift without retraining. By leveraging the invariant property of the likelihood ratio and the Bregman-divergence connection to posterior calibration, OBIL trains an ensemble of Bregman-calibrated networks on associated problems and online-adjusts decision thresholds using unlabeled data, with finite-sample regret $O\left(\sqrt{T \log T}\right)$. The approach combines a theoretically grounded offline LR estimator with a robust online prior-tracking mechanism, augmented by stability and calibration checks, and demonstrates strong performance under severe prior shifts on benchmark and medical datasets. The results show that OBIL maintains robust F1 scores where traditional rebalancing or post-hoc methods fail, and provide practical guidance for calibration, hyperparameter choices, and deployment constraints. Together, these contributions advance principled, online imbalanced learning capable of handling deployment-time priors without labeled target data.
Abstract
Class imbalance remains a fundamental challenge in machine learning, where standard classifiers exhibit severe performance degradation in minority classes. Although existing approaches address imbalance through resampling or cost-sensitive learning during training, they require retraining or access to labeled target data when class distributions shift at deployment time, a common occurrence in real-world applications such as fraud detection, medical diagnosis, and anomaly detection. We present \textit{Online Bayesian Imbalanced Learning} (OBIL), a principled framework that decouples likelihood-ratio estimation from class-prior assumptions, enabling real-time adaptation to distribution shifts without model retraining. Our approach builds on the established connection between Bregman divergences and proper scoring rules to show that deep networks trained with such losses produce posterior probability estimates from which prior-invariant likelihood ratios can be extracted. We prove that these likelihood-ratio estimates remain valid under arbitrary changes in class priors and cost structures, requiring only a threshold adjustment for optimal Bayes decisions. We derive finite-sample regret bounds demonstrating that OBIL achieves $O(\sqrt{T \log T})$ regret against an oracle with perfect prior knowledge. Extensive experiments on benchmark datasets and medical diagnosis benchmarks under simulated deployment shifts demonstrate that OBIL maintains robust performance under severe distribution shifts, outperforming state-of-the-art methods in F1 Score when test distributions deviate significantly from the training conditions.