FairT2I: Mitigating Social Bias in Text-to-Image Generation via Large Language Model-Assisted Detection and Attribute Rebalancing

TL;DR

FairT2I introduces a training-free, bias-aware framework for text-to-image generation by factorizing the generative score into attribute-conditioned components and reweighting them with a bias-aware latent variable distribution. It combines LLM-based bias detection with attribute resampling to automatically infer and adjust latent attributes from prompts, enabling inference-time debiasing without retraining. The approach is augmented by an interactive UI that lets users inspect, modify, and regenerate images under customized attribute distributions, improving both societal bias mitigation and output diversity while maintaining fidelity. Comprehensive automatic and human evaluations demonstrate that FairT2I outperforms baseline debiasing methods in fairness and diversity and can steer distributions toward uniform or real-world statistics, offering practical impact for fair and diverse image synthesis.

Abstract

Text-to-image (T2I) models have advanced creative content generation, yet their reliance on large uncurated datasets often reproduces societal biases. We present FairT2I, a training-free and interactive framework grounded in a mathematically principled latent variable guidance formulation. This formulation decomposes the generative score function into attribute-conditioned components and reweights them according to a defined distribution, providing a unified and flexible mechanism for bias-aware generation that also subsumes many existing ad hoc debiasing approaches as special cases. Building upon this foundation, FairT2I incorporates (1) latent variable guidance as the core mechanism, (2) LLM-based bias detection to automatically infer bias-prone categories and attributes from text prompts as part of the latent structure, and (3) attribute resampling, which allows users to adjust or redefine the attribute distribution based on uniform, real-world, or user-specified statistics. The accompanying user interface supports this pipeline by enabling users to inspect detected biases, modify attributes or weights, and generate debiased images in real time. Experimental results show that LLMs outperform average human annotators in the number and granularity of detected bias categories and attributes. Moreover, FairT2I achieves superior performance to baseline models in both societal bias mitigation and image diversity, while preserving image quality and prompt fidelity.

Figures (15)

• Figure 1: Overview of FairT2I. The LLM detects biases that exist in the input prompt and converts it into a set of bias-aware prompts by sampling the detected attributes from fair distributions. The interactive interface allows users to inspect, edit, and regenerate images, enabling both societal bias mitigation and improved diversity.
• Figure 2: Comparison of generation processes without (left) and with (right) latent variable guidance for the input prompt $\mathbf{y} = \textit{A CEO}$ and attribute set $\mathcal{Z} = \{z_1=\textit{male}, z_2=\textit{female}\}$. In the standard diffusion process, the model internalizes societal biases from the training data and denoises images toward male CEO representations, resulting in biased outputs. Latent variable guidance enables the adjustment of the denoising direction based on a fair attribute distribution, thereby allowing the mitigation of such biases in the generated results.
• Figure 3: Number of detected categories and attributes per category for the Stable Bias (left) and Parti Prompt (right) datasets. Bars show Human Mean (blue), Human Union (orange), and LLM (green), with error bars indicating one standard deviation.
• Figure 4: Generated images for the input text "an airplane flying into a cloud that looks like monster." by classifier-free guidance (CFG) at guidance scales 7.0, 4.0, and 1.0 and FairT2I (Ours). A guidance scale of 1.0 corresponds to generation without CFG.
• Figure 5: Our web application for bias-aware image generation. (1) Users first input a text prompt for image generation. (2) The LLM detects potential societal biases or implicit characteristics and presents them in a tabular format. (3) Users can edit the table to adjust categories or attributes as desired. (4) The T2I model then generates images based on the edited table, enabling debiased image generation. Please also refer to the supplemental video.

Theorems & Definitions (4)

• Proposition 1
• proof
• Proposition 1
• proof