Epistemic Uncertainty-Weighted Loss for Visual Bias Mitigation

TL;DR

This paper tackles visual bias mitigation when bias information is unavailable, by leveraging a fully Bayesian neural network and a predictive uncertainty-weighted loss. The core idea is to compute the predictive posterior for each sample via Monte Carlo sampling to obtain an epistemic uncertainty, and to weight each training example's loss accordingly with $\\hat{L}(x_i, y_i) = L(x_i, y_i)\\times(1.0 + \\sigma_{i, y_i})^{\\kappa}$, where $\\kappa$ controls the emphasis on high-uncertainty samples. The method uses Stochastic Gradient Langevin Dynamics (SGLD) and its cyclical variant (cSG-MCMC) to approximate the posterior and estimates uncertainty during training, with a short initial phase to stabilize uncertainties and caching to speed up computation. Empirical evaluation on CIFAR-10S (sensitive attribute bias) and a minority-group bias variant (CIFAR-10M), plus a real-world face-detection task with CelebA and FairFace, shows that the uncertainty-weighted approach can reduce bias indicators and TPR gaps, though it does not consistently outperform bias-informed baselines and can be sensitive to hyperparameters like \\(\\kappa\\). Overall, the work provides a principled, bias-agnostic mechanism for mitigating visual bias via epistemic uncertainty, with clear avenues for improvement and broader validation.

Abstract

Deep neural networks are highly susceptible to learning biases in visual data. While various methods have been proposed to mitigate such bias, the majority require explicit knowledge of the biases present in the training data in order to mitigate. We argue the relevance of exploring methods which are completely ignorant of the presence of any bias, but are capable of identifying and mitigating them. Furthermore, we propose using Bayesian neural networks with a predictive uncertainty-weighted loss function to dynamically identify potential bias in individual training samples and to weight them during training. We find a positive correlation between samples subject to bias and higher epistemic uncertainties. Finally, we show the method has potential to mitigate visual bias on a bias benchmark dataset and on a real-world face detection problem, and we consider the merits and weaknesses of our approach.

Figures (4)

• Figure 1: Face training samples from CelebA with lowest predictive uncertainties (left) and faces with highest predictive uncertainties (right). The faces with low uncertainties tend to be well-lit, facing forward with hair cleanly framing the face, and primarily lighter-skinned with few obscuring accessories. Faces with high uncertainties are more likely to be subject to discrimination due to variance in lighting, pose, colouring, and obscuring objects, among other reasons.
• Figure 2: Left: a random selection of samples from CelebA; Right: and the same from FairFaces.
• Figure 3: Performance discrepancies between baseline (deterministic model with no de-biasing) and Bayesian model with uncertainty-weighted loss for minority subgroups with lowest TPRs.
• Figure 4: True positive rate (TPR) over the entire FairFace dataset as a function of tunable de-biasing parameter kappa $\kappa$, showing how the degree of de-biasing can be controlled by $\kappa$.