Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CAD-VAE: Leveraging Correlation-Aware Latents for Comprehensive Fair Disentanglement

Chenrui Ma, Xi Xiao, Tianyang Wang, Xiao Wang, Yanning Shen

TL;DR

CAD-VAE introduces a correlation-aware disentanglement framework by incorporating a correlated latent code $z_R$ to capture information shared between target $Y$ and sensitive $S$ attributes. It directly minimizes the conditional mutual information $I(Y; S \,|\, z_R)$ and employs a relevance-learning strategy to ensure $z_R$ encodes only essential shared information, preserving discriminative power in $z_Y$ and $z_S$. The method combines a VAE objective with a CMI loss, a total correlation penalty, and a learning-relevance loss, optimized in a two-step process. Across fair classification, counterfactual generation, and fair image/text editing tasks, CAD-VAE achieves state-of-the-art fairness while maintaining predictive and generative quality, and demonstrates strong applicability to vision-language models via CLIP-based adaptations.

Abstract

While deep generative models have significantly advanced representation learning, they may inherit or amplify biases and fairness issues by encoding sensitive attributes alongside predictive features. Enforcing strict independence in disentanglement is often unrealistic when target and sensitive factors are naturally correlated. To address this challenge, we propose \textbf{CAD-VAE} (\textbf{C}orrelation-\textbf{A}ware \textbf{D}isentangled \textbf{VAE}), which introduces a correlated latent code to capture the information shared between the target and sensitive attributes. Given this correlated latent, our method effectively separates overlapping factors without extra domain knowledge by directly minimizing the conditional mutual information between target and sensitive codes. A relevance-driven optimization strategy refines the correlated code by efficiently capturing essential correlated features and eliminating redundancy. Extensive experiments on benchmark datasets demonstrate that CAD-VAE produces fairer representations, realistic counterfactuals, and improved fairness-aware image editing. Source code is available : https://github.com/merry7cherry/CAD-VAE

CAD-VAE: Leveraging Correlation-Aware Latents for Comprehensive Fair Disentanglement

TL;DR

CAD-VAE introduces a correlation-aware disentanglement framework by incorporating a correlated latent code to capture information shared between target and sensitive attributes. It directly minimizes the conditional mutual information and employs a relevance-learning strategy to ensure encodes only essential shared information, preserving discriminative power in and . The method combines a VAE objective with a CMI loss, a total correlation penalty, and a learning-relevance loss, optimized in a two-step process. Across fair classification, counterfactual generation, and fair image/text editing tasks, CAD-VAE achieves state-of-the-art fairness while maintaining predictive and generative quality, and demonstrates strong applicability to vision-language models via CLIP-based adaptations.

Abstract

While deep generative models have significantly advanced representation learning, they may inherit or amplify biases and fairness issues by encoding sensitive attributes alongside predictive features. Enforcing strict independence in disentanglement is often unrealistic when target and sensitive factors are naturally correlated. To address this challenge, we propose \textbf{CAD-VAE} (\textbf{C}orrelation-\textbf{A}ware \textbf{D}isentangled \textbf{VAE}), which introduces a correlated latent code to capture the information shared between the target and sensitive attributes. Given this correlated latent, our method effectively separates overlapping factors without extra domain knowledge by directly minimizing the conditional mutual information between target and sensitive codes. A relevance-driven optimization strategy refines the correlated code by efficiently capturing essential correlated features and eliminating redundancy. Extensive experiments on benchmark datasets demonstrate that CAD-VAE produces fairer representations, realistic counterfactuals, and improved fairness-aware image editing. Source code is available : https://github.com/merry7cherry/CAD-VAE

Paper Structure

This paper contains 38 sections, 4 theorems, 75 equations, 10 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Fair Disentanglement Learning
  4. Correlation-Aware Learning
  5. Counterfactual Fairness
  6. Preliminary
  7. Conditional Independence and Mutual Information
  8. Variational Autoencoder
  9. Total Correlation Loss
  10. Method
  11. CAD-VAE
  12. Conditional Independence for Disentanglement
  13. Direct Minimization of Conditional Mutual Information
  14. Learning Relevance Between Target And Sensitive Information
  15. Final Objective Function
  16. ...and 23 more sections

Key Result

Proposition 1

Let $A$, $B$, and $C$ be random variables. We say that $A$ is conditionally independent of $B$ given $C$, denoted $A \perp B \mid C$, if and only if their conditional joint probability distribution factorizes as follows:

Figures (10)

  • Figure 1: Illustration of the data flow. The orange lines connect the information in the observed space and their corresponding latent codes.
  • Figure 2: Examples of Fair Counterfactual Generation. Zoom in to check. The first row shows the source and reference images. Rows 2–5 display counterfactuals obtained by replacing latent subspaces $z_X$, $z_Y$, $z_S$, and $[z_S, z_R]$, respectively. Notably, the replacement with $[z_S, z_R]$ (row 5) naturally adapts sensitive features for different sensitive attributes without domain knowledge. (mustache for men and makeup for women).
  • Figure 3: Examples of Fair Fine-Grained Image Editing. Zoom in to check. The leftmost column shows the source and reference images. The blue-framed section displays images generated by interpolating $z_Y$ and $z_S$ (with $z_R$ and $z_X$ fixed), where the horizontal axis varies $z_S$ and the vertical axis varies $z_Y$. The red-framed section illustrates images produced by interpolating $z_Y$ and $z_R$ (with $z_S$ fully replaced by the reference and $z_X$ constant). Modification in one latent code minimally affecting others, harness $z_R$ to edit sensitive relevant feature(makeup or mustache).
  • Figure 4: Style transfer using StyleCLIP and the CAD-VAE extension. This example transforms the (a) into "a dancer with long blonde hair." "StyleCLIP+" means StyleCLIP + CAD-VAE.
  • Figure 5: Top: Impact of varying $\lambda_{CMI}$ (with $\lambda_{LRI}$ fixed at 60) on classification accuracy and fairness violation (EOD) in the CelebA dataset classification task. Bottom: Impact of varying $\lambda_{LRI}$ (with $\lambda_{CMI}$ fixed at 5) on the same performance metrics.
  • ...and 5 more figures

Theorems & Definitions (5)

  • Proposition 1: Conditional Independence
  • Definition 1: Conditional Mutual Information
  • Lemma 1: Properties of CMI
  • Lemma 2: Symmetry of CMI
  • Proposition 2