Marginal Debiased Network for Fair Visual Recognition

TL;DR

A novel marginal debiased network (MDN) is proposed to learn debiased representations by introducing the idea of margin penalty into the fairness problem, which assigns a larger margin for bias-conflicting samples than for bias-aligned samples so as to deemphasize the spurious correlations and improve generalization on unbiased test criteria.

Abstract

Deep neural networks (DNNs) are often prone to learn the spurious correlations between target classes and bias attributes, like gender and race, inherent in a major portion of training data (bias-aligned samples), thus showing unfair behavior and arising controversy in the modern pluralistic and egalitarian society. In this paper, we propose a novel marginal debiased network (MDN) to learn debiased representations. More specifically, a marginal softmax loss (MSL) is designed by introducing the idea of margin penalty into the fairness problem, which assigns a larger margin for bias-conflicting samples (data without spurious correlations) than for bias-aligned ones, so as to deemphasize the spurious correlations and improve generalization on unbiased test criteria. To determine the margins, our MDN is optimized through a meta learning framework. We propose a meta equalized loss (MEL) to perceive the model fairness, and adaptively update the margin parameters by meta-optimization which requires the trained model guided by the optimal margins should minimize MEL computed on an unbiased meta-validation set. Extensive experiments on BiasedMNIST, Corrupted CIFAR-10, CelebA and UTK-Face datasets demonstrate that our MDN can achieve a remarkable performance on under-represented samples and obtain superior debiased results against the previous approaches.

Figures (5)

• Figure 1: The vanilla model learns the spurious correlations from bias-aligned samples (short-haired men) while ignoring bias-conflicting samples (short-haired women), leading to a biased problem (upper). Our MDN introduces a margin penalty and leverages meta learning to assign stronger constrains for bias-conflicting samples, which achieves fair performance when evaluated on unbiased datasets (lower).
• Figure 2: Illustration of MDN. First, we perform representation learning on training data, which contains two main steps. i) Forward process to compute the marginal softmax loss in Eq. (\ref{['source']}); ii) Backward process to update the network parameters $\theta$ and $\phi \left ( m \right )$ using Eq. (\ref{['6']}), where $m$ is variable used for parameterizing $\phi$. Second, we perform margin learning on meta-validation data, including two steps. iii) Forward process to compute the meta equalized loss according to Eq. (\ref{['MEL']}); iv) Backward-on-backward to update the margin parameter $m$ using Eq. (\ref{['opt_margin']}).
• Figure 3: The comparison of decision boundaries for different loss functions in the binary-classes scenarios. Red dashed line represents the decision boundary when training, and the gray one is the decision boundary when testing. (We assume that the images from target class 0 are bias-conflicting samples when $b=0$ and the opposite when $b=1$.)
• Figure 4: t-SNE maaten2008visualizing embedding visualizations on CelebA when mitigating gender bias. Here Y and B respectively represent target and bias attributes. Y1B0 denotes the group of the images with target label $y=1$ and bias label $b=0$, and similarly for Y0B0, Y0B1 and Y1B1, respectively. The number of training images in each group is shown in the bracket.
• Figure 5: The margin parameters learned for different groups on CelebA when mitigating gender bias. Here Y and B respectively represent target and bias attributes. $m_{1,0}$ denotes the margin parameter learned for the images with target label $y=1$ and bias label $b=0$, and similarly for $m_{0,0}$, $m_{0,1}$ and $m_{1,1}$, respectively. The number of training images in each group is shown in the bracket.