Improving Fairness and Mitigating MADness in Generative Models

TL;DR

This work targets bias in generative modeling by linking MLE-induced estimator bias to fairness gaps and to Model Autophagy Disorder (MADness). It introduces Penalized Autophagy Estimation (PLE) implemented via hypernetworks to enforce that parameter estimates from real and synthetic data share consistent statistics, thereby reducing bias and stabilizing generation. Theoretical formulation pairs a constrained objective with a tractable Lagrangian relaxation and a data-driven hypernetwork $H_{\phi}$ that predicts downstream weights, enabling scalable debiasing across architectures. Empirically, hypernetwork-enabled PLE improves minority-data representation, slows MADness, and yields more robust performance in imbalanced or low-data regimes across MNIST, GMM, and CIFAR-10 settings. Overall, the approach couples unbiased statistical estimation with deep learning to enhance fairness and stability in generative modelling, with potential extensions to diffusion methods and uncertainty quantification.

Abstract

Generative models unfairly penalize data belonging to minority classes, suffer from model autophagy disorder (MADness), and learn biased estimates of the underlying distribution parameters. Our theoretical and empirical results show that training generative models with intentionally designed hypernetworks leads to models that 1) are more fair when generating datapoints belonging to minority classes 2) are more stable in a self-consumed (i.e., MAD) setting, and 3) learn parameters that are less statistically biased. To further mitigate unfairness, MADness, and bias, we introduce a regularization term that penalizes discrepancies between a generative model's estimated weights when trained on real data versus its own synthetic data. To facilitate training existing deep generative models within our framework, we offer a scalable implementation of hypernetworks that automatically generates a hypernetwork architecture for any given generative model.

Figures (5)

• Figure 1: The self-consuming parameter estimation loop.
• Figure 2: PLE is more stable and outperforms the baseline as we train models on their own outputs (MADness). This plot shows the generation versus FID for BigGAN trained on CIFAR-10. The baseline (labeled BigGAN) uses normal BigGAN training and collapses after only three generations. On the other hand, our method is very stable and only sees a slight increase in FID over the course of the three generations.
• Figure 3: Comparison of KL divergence differences, i.e., $\mathbb{D}_{KL}(q_{\,\text{MLE}} \mid\mid p_{\,\text{GMM}}) - \mathbb{D}_{KL}(q_{\,\text{PLE}} \mid\mid p_{\,\text{GMM}})$, for estimating GMM parameters. Positive values indicate scenarios where PLE outperforms MLE, particularly in imbalanced datasets and low-data regimes.
• Figure 4: MLE vs PLE Estimates of the parameters of various distributions. Notice how MLE collapses into MADNess much faster than PLE. More details can be found in Section \ref{['sec:madcow_grid_explanation']}
• Figure 5: Generation index versus MLE and PLE of standard deviation for standard normal, $U[0, 1]$. The error bars display the standard error for $100$ different initializations. These results use the analytic form of the Gaussian standard deviation derived in Section \ref{['sec:gaussian_std_estimation']}. For a data-driven version of this plot, see Figure \ref{['fig:madcow_grid_fig']}.