Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Provable Optimization for Adversarial Fair Self-supervised Contrastive Learning

Qi Qi, Quanqi Hu, Qihang Lin, Tianbao Yang

TL;DR

The paper tackles fair representation learning in a self-supervised setting by formulating a non-convex minimax problem that couples a global contrastive loss with an adversarial fairness regularizer. It introduces SoFCLR, a stochastic algorithm using moving-average estimators to handle the compositional gradient structure and prove convergence to an $\epsilon$-stationary point under mild assumptions. The approach preserves the flexibility of SSL losses while enforcing distributional fairness, as evidenced by improved fairness metrics across CelebA and UTKFace with partial attribute labeling. The results suggest practical utility for deploying fair SSL encoders in real-world, label-scarce scenarios, though extending to multi-modality remains an avenue for future work.

Abstract

This paper studies learning fair encoders in a self-supervised learning (SSL) setting, in which all data are unlabeled and only a small portion of them are annotated with sensitive attribute. Adversarial fair representation learning is well suited for this scenario by minimizing a contrastive loss over unlabeled data while maximizing an adversarial loss of predicting the sensitive attribute over the data with sensitive attribute. Nevertheless, optimizing adversarial fair representation learning presents significant challenges due to solving a non-convex non-concave minimax game. The complexity deepens when incorporating a global contrastive loss that contrasts each anchor data point against all other examples. A central question is ``{\it can we design a provable yet efficient algorithm for solving adversarial fair self-supervised contrastive learning}?'' Building on advanced optimization techniques, we propose a stochastic algorithm dubbed SoFCLR with a convergence analysis under reasonable conditions without requring a large batch size. We conduct extensive experiments to demonstrate the effectiveness of the proposed approach for downstream classification with eight fairness notions.

Provable Optimization for Adversarial Fair Self-supervised Contrastive Learning

TL;DR

The paper tackles fair representation learning in a self-supervised setting by formulating a non-convex minimax problem that couples a global contrastive loss with an adversarial fairness regularizer. It introduces SoFCLR, a stochastic algorithm using moving-average estimators to handle the compositional gradient structure and prove convergence to an -stationary point under mild assumptions. The approach preserves the flexibility of SSL losses while enforcing distributional fairness, as evidenced by improved fairness metrics across CelebA and UTKFace with partial attribute labeling. The results suggest practical utility for deploying fair SSL encoders in real-world, label-scarce scenarios, though extending to multi-modality remains an avenue for future work.

Abstract

This paper studies learning fair encoders in a self-supervised learning (SSL) setting, in which all data are unlabeled and only a small portion of them are annotated with sensitive attribute. Adversarial fair representation learning is well suited for this scenario by minimizing a contrastive loss over unlabeled data while maximizing an adversarial loss of predicting the sensitive attribute over the data with sensitive attribute. Nevertheless, optimizing adversarial fair representation learning presents significant challenges due to solving a non-convex non-concave minimax game. The complexity deepens when incorporating a global contrastive loss that contrasts each anchor data point against all other examples. A central question is ``{\it can we design a provable yet efficient algorithm for solving adversarial fair self-supervised contrastive learning}?'' Building on advanced optimization techniques, we propose a stochastic algorithm dubbed SoFCLR with a convergence analysis under reasonable conditions without requring a large batch size. We conduct extensive experiments to demonstrate the effectiveness of the proposed approach for downstream classification with eight fairness notions.
Paper Structure (22 sections, 8 theorems, 51 equations, 4 figures, 7 tables, 1 algorithm)

This paper contains 22 sections, 8 theorems, 51 equations, 4 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. The Formulation and Justification
  5. Stochastic Algorithm and Analysis
  6. Algorithm Design
  7. Convergence Analysis
  8. Experiments
  9. Prediction Results
  10. Fair representation visualization
  11. Effectiveness of fairness regularizer
  12. Conclusions
  13. Fairness Verification
  14. Convergence Analysis of SoFCLR
  15. Proof of Lemma \ref{['lem:1']}
  16. ...and 7 more sections

Key Result

Proposition 1

Suppose $E$ and $D$ have enough capacity, then the global optimal solution to the zero-sum game $\min_{E}\max_{D}F_{\text{fair}}(E, D)$ denoted by $E_*, D_*$ would satisfy $p(E_*({\bf{x}})|a) = p(E_*({\bf{x}}))$ and $[D_*(E_*({\bf{x}}))]_k = p(a=k)$.

Figures (4)

  • Figure 1: Learned representations of 1000 testing examples from CelebA by different methods.
  • Figure 2: SoFCLR accuracy vs $\Delta$ ED balance (left), and adversarial loss evolution with varying $\alpha$ (right) on UTKFace data.
  • Figure 3: The convergence curves of different objective components optimized by SoFCLR with varying $\alpha$ values on the UTKFace dataset, using gender as the target label and different sensitive attributes, are shown in the figure.
  • Figure 4: Prediction score distributions for positive and negative class on UTKFace with gender as the target and ethnicity as the sensitive attribute.

Theorems & Definitions (13)

  • Definition 1: Distributional Representation Fairness
  • Proposition 1
  • Theorem 2
  • proof : Proof of Theorem \ref{['thm:fair_verification']}
  • Lemma 1
  • Lemma 2
  • Lemma 3: Lemma 9 in qiu2023largescale
  • Lemma 4
  • Theorem 3
  • Lemma 5
  • ...and 3 more