When majority rules, minority loses: bias amplification of gradient descent

TL;DR

The paper addresses bias amplification in gradient-based learning under strong population and variance imbalance by formalizing majority-minority tasks with a two-term loss $L = L_1 + L_0$ and proving a Kantorovich-type perturbation relation between the full-data predictor and the majority-only predictor. It shows that critical points of $L$ are near corresponding points of $L_1$, with a stereotype gap bounded by $\|\hat{\theta}_1-\hat{\theta}\| \le 4\tau/\delta$, and that gradient descent trajectories spend most of their time in a majority-training regime before minority features influence predictions. The work also derives lower bounds on the debiasing duration and the catch-up overcost required to achieve minority-aware predictions, and validates the theory with numerical experiments on tabular and image datasets (e.g., CIFAR-10, EuroSAT, Adult). These results illuminate why bias amplification occurs and suggest practical implications: longer training and/or larger models can foster minority awareness, albeit with risks of spurious correlations, guiding fairer deployment of ML systems.

Abstract

Despite growing empirical evidence of bias amplification in machine learning, its theoretical foundations remain poorly understood. We develop a formal framework for majority-minority learning tasks, showing how standard training can favor majority groups and produce stereotypical predictors that neglect minority-specific features. Assuming population and variance imbalance, our analysis reveals three key findings: (i) the close proximity between ``full-data'' and stereotypical predictors, (ii) the dominance of a region where training the entire model tends to merely learn the majority traits, and (iii) a lower bound on the additional training required. Our results are illustrated through experiments in deep learning for tabular and image classification tasks.

Figures (8)

• Figure 1: Stereotypical and representative predictions for imbalanced CIFAR-2 ($n_0/n \approx 3\%$, see \ref{['sec:datasets_detail']}) with ResNet 18. We report the average and standard deviation over 30 runs.
• Figure 2: Training curves on 'Imbalanced CIFAR 2' with ResNet18 (see \ref{['sec:train-metrics']}). Left to right: unlucky curve with stopping rule based on minority recognition, i.e., $\text{Acc}_0 > 99\%$; random trajectory with the same rule; random trajectory with global accuracy stopping rule $\text{Acc} > 99\%$. Unlucky initialization has 600 epochs while 'careless training' (third one) needs 100 epochs and has much higher final $L_0$ value. Middle: random initialization with minority aware stopping rule training has 140 epochs. In the real world $\text{Acc}_0 > 99\%$ is not a realistic criterion as we do not know the minority class. Conclusion: risk-averse training should rely on considerably longer training (here +500%), confirming the results of \ref{['sec:train-phase']}. For more confident training, substantially longer runs are still required (here +40%).
• Figure 3: Training accuracy for different subgroup imbalance scenarios (1%, 10%, and 30%) using ResNet18 on CIFAR-10 and threshold $\kappa = 90\%$. Greater imbalance delays the learning of minority features: their accuracy reaches $\kappa$ later.
• Figure 4: Training and test loss/accuracy for ResNet-18 on EuroSAT (mean of 3 runs). Minority classes exhibit delayed learning -- their accuracy improves substantially only in later epochs.
• Figure 5: Loss and accuracy with TabNet on Adult (mean of 3 runs). Under strong imbalance, the catch-up overcost is substantial --- around 400%.

Theorems & Definitions (35)

• Theorem 1: Strong imbalance and critical points
• Corollary 1: Distances between critical and local minimizer sets
• Theorem 2: Majority adverse zone
• Remark 1: On the majority-training zone size
• Lemma 1: The majority adverse zone favors minority
• Proposition 1: Training duration
• Example 1
• Proposition 2: Debiasing duration
• Example 1: continued
• Lemma 2