Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Equalized Generative Treatment: Matching f-divergences for Fairness in Generative Models

Alexandre Verine, Rafael Pinot, Florian Le Bronnec

TL;DR

The paper addresses fairness in generative modeling, arguing that proportion-based criteria like EGO and MGO are brittle because they do not guarantee equal conditional generation quality across sensitive groups. It defines equalized generative treatment (EGT) via δ-EGT, balancing group-wise $f$-divergences $\mathcal{D}_f(P_a\|Q_a)$, and shows that enforcing EGT relates to minimizing the worst-group divergence. The authors propose a practical min–max training approach to enforce EGT and provide theoretical results bounding the global divergence by the worst conditional divergence. Empirically, Min–Max improves per-group fairness metrics across diffusion-based image generation and text generation with competitive overall performance, while highlighting trade-offs and the limited predictive power of proportion-based metrics for conditional quality. This work offers a principled framework and a scalable optimization strategy for fairness in generative systems, with potential extensions to other divergences and modalities.

Abstract

Fairness is a crucial concern for generative models, which not only reflect but can also amplify societal and cultural biases. Existing fairness notions for generative models are largely adapted from classification and focus on balancing the probability of generating samples from each sensitive group. We show that such criteria are brittle, as they can be met even when different sensitive groups are modeled with widely varying quality. To address this limitation, we introduce a new fairness definition for generative models, termed as equalized generative treatment (EGT), which requires comparable generation quality across all sensitive groups, with quality measured via a reference f-divergence. We further analyze the trade-offs induced by EGT, demonstrating that enforcing fairness constraints necessarily couples the overall model quality to that of the most challenging group to approximate. This indicates that a simple yet efficient min-max fine-tuning method should be able to balance f-divergences across sensitive groups to satisfy EGT. We validate this theoretical insight through a set of experiments on both image and text generation tasks. We demonstrate that min-max methods consistently achieve fairer outcomes compared to other approaches from the literature, while maintaining competitive overall performance for both tasks.

Equalized Generative Treatment: Matching f-divergences for Fairness in Generative Models

TL;DR

The paper addresses fairness in generative modeling, arguing that proportion-based criteria like EGO and MGO are brittle because they do not guarantee equal conditional generation quality across sensitive groups. It defines equalized generative treatment (EGT) via δ-EGT, balancing group-wise -divergences , and shows that enforcing EGT relates to minimizing the worst-group divergence. The authors propose a practical min–max training approach to enforce EGT and provide theoretical results bounding the global divergence by the worst conditional divergence. Empirically, Min–Max improves per-group fairness metrics across diffusion-based image generation and text generation with competitive overall performance, while highlighting trade-offs and the limited predictive power of proportion-based metrics for conditional quality. This work offers a principled framework and a scalable optimization strategy for fairness in generative systems, with potential extensions to other divergences and modalities.

Abstract

Fairness is a crucial concern for generative models, which not only reflect but can also amplify societal and cultural biases. Existing fairness notions for generative models are largely adapted from classification and focus on balancing the probability of generating samples from each sensitive group. We show that such criteria are brittle, as they can be met even when different sensitive groups are modeled with widely varying quality. To address this limitation, we introduce a new fairness definition for generative models, termed as equalized generative treatment (EGT), which requires comparable generation quality across all sensitive groups, with quality measured via a reference f-divergence. We further analyze the trade-offs induced by EGT, demonstrating that enforcing fairness constraints necessarily couples the overall model quality to that of the most challenging group to approximate. This indicates that a simple yet efficient min-max fine-tuning method should be able to balance f-divergences across sensitive groups to satisfy EGT. We validate this theoretical insight through a set of experiments on both image and text generation tasks. We demonstrate that min-max methods consistently achieve fairer outcomes compared to other approaches from the literature, while maintaining competitive overall performance for both tasks.
Paper Structure (40 sections, 6 theorems, 43 equations, 5 figures, 8 tables, 2 algorithms)

This paper contains 40 sections, 6 theorems, 43 equations, 5 figures, 8 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background & Related Works
  3. $f$-divergences in generative modeling
  4. Fairness in Generative Modeling
  5. On the brittleness of existing definitions
  6. Theoretical Brittleness of Matching and Equalized Generative Odds
  7. Observing the brittleness in practice
  8. Equalized Generative Treatment (EGT)
  9. Definition and First Property
  10. Min-Max as Candidate for Enforcing EGT
  11. Improved fairness through EGT
  12. Concluding Remarks
  13. Mathematical Supplementary Material
  14. Useful Lemma on the Surjectivity of $\mathcal{D}_f$
  15. Additional sanity checks for the construction of $Q_\alpha^\beta$ in Lemma \ref{['lemma:surjectivity']}
  16. ...and 25 more sections

Key Result

Theorem 3.1

Let $P \in \mathcal{P}_\lambda \left( \mathcal{X} \right)$ be a non-trivial target distribution satisfying EGO, and let $f$ be a continuous function such that $\mathcal{D}_f$ defines an $f$-divergence. For any $\epsilon \in (0, f(0) + \bar{f}(+\infty))$ and any $\gamma \in (0,\epsilon)$, there exist

Figures (5)

  • Figure 1: Jensen–Shannon divergence between a target distribution $P$ and rescaled Gaussian models $Q=\mathcal{N}(\mu,\sigma^2)$. (\ref{['fig:brittle1']}) Models with the same global divergence $\mathcal{D}_{\mathrm{JS}}(P\Vert Q)$ can still differ greatly. (\ref{['fig:brittle2']}) Level set for $\mathcal{D}_{\mathrm{JS}}=1$, with selected models marked by stars. (\ref{['fig:brittle3']})--(\ref{['fig:brittle4']}) Conditional divergences for the two groups, models on the same level set may yield highly unbalanced conditional divergences.
  • Figure 2: Precision and recall for EDM (VE and VP) on FFHQ (\ref{['fig:ffhq_mgo_vp']}) and for LLaMA-3.2-Chat (1B and 3B) and Gemma-3 4B (pretrained and instructed-tuned) on the Wikipedia Biographies (\ref{['fig:llama_chat']}) under two settings: pretrained and conditional. At the sampling stage, rejection sampling is used to enforce either MGO or EGO. Each sensitive-group is color coded (red or blue) and the points corresponding to the subgroups for a given model are connected to each other by a dashed line. We observe significant discrepancies in precision and recall persist across groups, demonstrating the brittleness of proportion-based definitions.
  • Figure 3: Estimated denoising losses per noise level for EDM-VP trained on FFHQ. Baseline exhibits a persistent gap between male and female groups. Reweighting and Min–Max reduce the gap, while conditional training almost eliminates it.
  • Figure 4: Samples from distribution of FFHQ faces with similar precision ($79.06$ (\ref{['app:fig:cata:all']}) and $77.12$ (\ref{['app:fig:cata:male']})) and similar recall $R$ ($60.02$ (\ref{['app:fig:cata:all']}) and $60.31$ (\ref{['app:fig:cata:male']})). However, the precision and recall for the sub-group are very different. In particular, in Example 1 the models generates slighly noised images for both classes. In Example 2, the model generates limited diversity and noisy images for Male Class while it generates high quality images and diverse samples for Female.
  • Figure 5: Comparison of the estimated loss for the baseline, Min-Max, conditional and reweighted methods on the FFHQ dataset with the VE model. The loss is plotted for every noise level.

Theorems & Definitions (14)

  • Definition 2.1
  • Definition 2.2
  • Theorem 3.1
  • Definition 4.1
  • Definition 4.2
  • Theorem 4.3
  • Lemma 1.1
  • proof
  • Lemma 1.2
  • proof
  • ...and 4 more