Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Self-Supervised Learning Pipeline for Demographically Fair Facial Attribute Classification

Sreeraj Ramachandran, Ajita Rattani

TL;DR

This work tackles demographic bias in facial attribute classification by shifting from supervised bias mitigation to a fully self-supervised learning pipeline that leverages unlabeled data. It combines a data curation step, zero-shot pseudo-labeling via encoders like CLIP, and a SupCon-based contrastive learning framework with meta-weight learning to balance accuracy and fairness without explicit sensitive labels. The approach demonstrates strong gains on FairFace and CelebA, outperforming standard SSL baselines and approaching or exceeding supervised-from-scratch performance while improving fairness metrics. This has practical significance for scalable, privacy-conscious fair facial analysis in real-world deployments where labeled demographic data are scarce or sensitive.

Abstract

Published research highlights the presence of demographic bias in automated facial attribute classification. The proposed bias mitigation techniques are mostly based on supervised learning, which requires a large amount of labeled training data for generalizability and scalability. However, labeled data is limited, requires laborious annotation, poses privacy risks, and can perpetuate human bias. In contrast, self-supervised learning (SSL) capitalizes on freely available unlabeled data, rendering trained models more scalable and generalizable. However, these label-free SSL models may also introduce biases by sampling false negative pairs, especially at low-data regimes 200K images) under low compute settings. Further, SSL-based models may suffer from performance degradation due to a lack of quality assurance of the unlabeled data sourced from the web. This paper proposes a fully self-supervised pipeline for demographically fair facial attribute classifiers. Leveraging completely unlabeled data pseudolabeled via pre-trained encoders, diverse data curation techniques, and meta-learning-based weighted contrastive learning, our method significantly outperforms existing SSL approaches proposed for downstream image classification tasks. Extensive evaluations on the FairFace and CelebA datasets demonstrate the efficacy of our pipeline in obtaining fair performance over existing baselines. Thus, setting a new benchmark for SSL in the fairness of facial attribute classification.

A Self-Supervised Learning Pipeline for Demographically Fair Facial Attribute Classification

TL;DR

This work tackles demographic bias in facial attribute classification by shifting from supervised bias mitigation to a fully self-supervised learning pipeline that leverages unlabeled data. It combines a data curation step, zero-shot pseudo-labeling via encoders like CLIP, and a SupCon-based contrastive learning framework with meta-weight learning to balance accuracy and fairness without explicit sensitive labels. The approach demonstrates strong gains on FairFace and CelebA, outperforming standard SSL baselines and approaching or exceeding supervised-from-scratch performance while improving fairness metrics. This has practical significance for scalable, privacy-conscious fair facial analysis in real-world deployments where labeled demographic data are scarce or sensitive.

Abstract

Published research highlights the presence of demographic bias in automated facial attribute classification. The proposed bias mitigation techniques are mostly based on supervised learning, which requires a large amount of labeled training data for generalizability and scalability. However, labeled data is limited, requires laborious annotation, poses privacy risks, and can perpetuate human bias. In contrast, self-supervised learning (SSL) capitalizes on freely available unlabeled data, rendering trained models more scalable and generalizable. However, these label-free SSL models may also introduce biases by sampling false negative pairs, especially at low-data regimes 200K images) under low compute settings. Further, SSL-based models may suffer from performance degradation due to a lack of quality assurance of the unlabeled data sourced from the web. This paper proposes a fully self-supervised pipeline for demographically fair facial attribute classifiers. Leveraging completely unlabeled data pseudolabeled via pre-trained encoders, diverse data curation techniques, and meta-learning-based weighted contrastive learning, our method significantly outperforms existing SSL approaches proposed for downstream image classification tasks. Extensive evaluations on the FairFace and CelebA datasets demonstrate the efficacy of our pipeline in obtaining fair performance over existing baselines. Thus, setting a new benchmark for SSL in the fairness of facial attribute classification.
Paper Structure (24 sections, 6 equations, 1 figure, 5 tables, 1 algorithm)

This paper contains 24 sections, 6 equations, 1 figure, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Method
  4. Data Curation Pipeline
  5. Self-supervised Training Pipeline
  6. Pseudolabeling using zero-shot techniques
  7. Supervised Contrastive Learning
  8. Meta-Weighted SupCon with Fairness Constraints
  9. Fairness Constraint
  10. Meta Weight Learning
  11. Experiments and Results
  12. Dataset
  13. Experimental Configuration and Metrics
  14. Evaluation on FairFace
  15. Cross Dataset Evaluation on CelebA
  16. ...and 9 more sections

Figures (1)

  • Figure 1: Overview of the proposed SSL framework. A) Data Curation Pipeline: Embeddings for unlabeled data are generated using a pre-trained encoder and used to deduplicate the data. The similarity between curated and noncurated embeddings is used to retrieve similar samples from the noncurated set to generate the final augmented curated set. (B) Training Pipeline: The encoder is trained using the augmented curated data and attribute labels obtained using zero-shot pseudo-labeled. SSL pretraining is performed using the SupCon loss, followed by weighted meta-learning to improve performance. Finally, a linear evaluation (probing) is conducted for the quality assessment of the learned embeddings.