FLAC: Fairness-Aware Representation Learning by Suppressing Attribute-Class Associations

TL;DR

FLAC proposes a sampling strategy that highlights underrepresented samples in the dataset, and casts the problem of learning fair representations as a probability matching problem that leverages representations extracted by a bias-capturing classifier, and it is theoretically shown that FLAC can indeed lead to fair representations, that are independent of the protected attributes.

Abstract

Bias in computer vision systems can perpetuate or even amplify discrimination against certain populations. Considering that bias is often introduced by biased visual datasets, many recent research efforts focus on training fair models using such data. However, most of them heavily rely on the availability of protected attribute labels in the dataset, which limits their applicability, while label-unaware approaches, i.e., approaches operating without such labels, exhibit considerably lower performance. To overcome these limitations, this work introduces FLAC, a methodology that minimizes mutual information between the features extracted by the model and a protected attribute, without the use of attribute labels. To do that, FLAC proposes a sampling strategy that highlights underrepresented samples in the dataset, and casts the problem of learning fair representations as a probability matching problem that leverages representations extracted by a bias-capturing classifier. It is theoretically shown that FLAC can indeed lead to fair representations, that are independent of the protected attributes. FLAC surpasses the current state-of-the-art on Biased-MNIST, CelebA, and UTKFace, by 29.1%, 18.1%, and 21.9%, respectively. Additionally, FLAC exhibits 2.2% increased accuracy on ImageNet-A and up to 4.2% increased accuracy on Corrupted-Cifar10. Finally, in most experiments, FLAC even outperforms the bias label-aware state-of-the-art methods.

Figures (5)

• Figure 1: Distances between the central sample and the rest of the samples belonging to Biased-MNIST, a dataset that demonstrates a strong association between the target labels (i.e., digits) and the protected attributes (i.e., colors). Subfigure on the left shows that using the standard task-specific loss introduces bias to the representations which FLAC, the proposed approach, can effectively mitigate as shown on the right. In particular, FLAC focuses on situations where task-specific losses are susceptible to data bias, namely pairs with the same target label but different protected attribute values (i.e., $\mathcal{A}_{10}$) and pairs with different target labels and the same protected attribute value (i.e., $\mathcal{A}_{01}$).
• Figure 2: Illustration of the proposed framework.
• Figure 3: The distributions of similarities or/and distances for the samples that satisfy Equation \ref{['eq:cond']}. Red color represents the sample pairs with the same target, but different protected attribute label, while blue depicts the pairs with different target, but the same protected attribute labels.
• Figure 4: The top 8 images that are most similar to queries representing a minority group of UTKFace (i.e., non-white males) based on the representations derived by Vanilla and FLAC approaches. Images with green borders denote the query samples.
• Figure 5: The frequency of the number of sample pairs belonging to $\mathcal{S}$ per batch ($N=128$) for Biased-MNIST training set with $p\in\{0.99,0.995,0.997,0.999\}$.