Happy Young Women, Grumpy Old Men? Emotion-Driven Demographic Biases in Synthetic Face Generation

TL;DR

The paper tackles the problem that text-to-image models generate faces with demographic and aesthetic biases, potentially amplified by emotional prompts. It conducts a cross-cultural audit of eight diffusion-based T2I models (four Western and four Chinese) using identical English prompts and evaluates gender, race, age, and attractiveness with FairFace and an attractiveness model, applying information-theoretic metrics like $D_{KL}$, $D_{JS}$, and TVD against global baselines and neutral prompts. The study finds persistent biases across models, with over-representation of White and young individuals and strong age-skew, and demonstrates that emotion-conditioned prompts systematically shift demographic distributions, often amplifying ageism; cross-cultural differences are smaller than expected, suggesting convergence toward Western-centric defaults. These findings have significant fairness and governance implications, underscoring the need for transparency, cross-cultural testing, and ongoing bias audits in generative AI systems. The authors propose a scalable bias-measurement framework and highlight practical steps for mitigating harms, including granular user controls and regulatory oversight.

Abstract

Synthetic face generation has rapidly advanced with the emergence of text-to-image (T2I) and of multimodal large language models, enabling high-fidelity image production from natural-language prompts. Despite the widespread adoption of these tools, the biases, representational quality, and cross-cultural consistency of these models remain poorly understood. Prior research on biases in the synthetic generation of human faces has examined demographic biases, yet there is little research on how emotional prompts influence demographic representation and how models trained in different cultural and linguistic contexts vary in their output distributions. We present a systematic audit of eight state-of-the-art T2I models comprising four models developed by Western organizations and four developed by Chinese institutions, all prompted identically. Using state-of-the-art facial analysis algorithms, we estimate the gender, race, age, and attractiveness levels in the generated faces. To measure the deviations from global population statistics, we apply information-theoretic bias metrics including Kullback-Leibler and Jensen-Shannon divergences. Our findings reveal persistent demographic and emotion-conditioned biases in all models regardless of their country of origin. We discuss implications for fairness, socio-technical harms, governance, and the development of transparent generative systems.

Figures (11)

• Figure 1: Exemplary faces generated by each model and for each emotion.
• Figure 2: Individual demographic distributions of neutral faces generated by the audited models (Western models in blue and Chinese models in red). From left to right, the graphs show the distributions of gender, race, and age, compared against the corresponding real-world population estimates, marked by green dashed horizontal lines. Upward (↑) and downward (↓) arrows denote over- and under-representation relative to the real-world statistics, respectively.
• Figure 3: Intersectional Age–Gender–Race distributions of the faces generated by Western (top) and Chinese (bottom) models.
• Figure 4: Emotion-induced demographic distribution shifts across models. KL divergence between emotion-conditioned and neutral demographic distributions is shown for different attributes. Emotion categories are ordered by the overall KL divergence averaged across models.
• Figure 5: Heatmaps of the distribution shifts (mean $\Delta P$) of facial attributes induced by emotional prompts across Western (above) and Chinese (below) models. Sur., Hap., Sad, Dis., and Ang. are short for Surprise, Happiness, Sadness, Disgust, and Anger, respectively.