RAIGen: Rare Attribute Identification in Text-to-Image Generative Models

TL;DR

RAIGen is introduced, the first framework, to the authors' knowledge, for un-supervised rare-attribute discovery in diffusion models, and leverages Matryoshka Sparse Autoencoders and a novel minority metric combining neuron activation frequency with semantic distinctiveness to identify interpretable neurons whose top-activating images reveal underrepresented attributes.

Abstract

Text-to-image diffusion models achieve impressive generation quality but inherit and amplify training-data biases, skewing coverage of semantic attributes. Prior work addresses this in two ways. Closed-set approaches mitigate biases in predefined fairness categories (e.g., gender, race), assuming socially salient minority attributes are known a priori. Open-set approaches frame the task as bias identification, highlighting majority attributes that dominate outputs. Both overlook a complementary task: uncovering rare or minority features underrepresented in the data distribution (social, cultural, or stylistic) yet still encoded in model representations. We introduce RAIGen, the first framework, to our knowledge, for un-supervised rare-attribute discovery in diffusion models. RAIGen leverages Matryoshka Sparse Autoencoders and a novel minority metric combining neuron activation frequency with semantic distinctiveness to identify interpretable neurons whose top-activating images reveal underrepresented attributes. Experiments show RAIGen discovers attributes beyond fixed fairness categories in Stable Diffusion, scales to larger models such as SDXL, supports systematic auditing across architectures, and enables targeted amplification of rare attributes during generation.

Figures (17)

• Figure 1: Overview of RAIGen. Diffusion representations ($\mathbf{h}$) are decomposed by MSAE into interpretable features ($\mathbf{z}$). A minority score ($s$), combining rarity and distinctiveness, ranks neurons or features to reveal minority attributes. Minority concepts are identified at the coarsest MSAE level (e.g., female doctor, doctor in suit), where size reflects activation frequency (smaller size = less frequent) and color denotes neuron identity.
• Figure 2: Least-active latents preferentially map to rare ground-truth features in the toy setting. The blue bars report $\mathbb{P}(\text{rare feature}\mid \text{latent}\in\text{least-active})$, i.e., the fraction of least-active latents whose matched feature is rare. The orange bars report the same probability for a random baseline, computed by sampling the same number of latents uniformly at random.
• Figure 3: WinoBias qualitative examples on SDXL. Top-activating images and MSAE activation heatmaps for minority neurons discovered by RAIGen across three WinoBias profession prompts: Doctor (top), Sheriff (middle), and Writer (bottom). Labels above each group show generated language annotations for the corresponding neuron.
• Figure 4: COCO qualitative examples on SDXL and SD v1.4. Top-activating images and MSAE activation heatmaps for minority neurons discovered by RAIGen for two COCO prompts: "A woman taking a picture of herself in front of a desktop" using SDXL (top row) and "A train going down a track at full speed" using SD v1.4 (bottom). Labels above each group indicate generated language annotation for the corresponding neuron.
• Figure 5: Long-tailed feature frequencies in the toy setting. Activation counts for each ground-truth feature over $N{=}300{,}000$ samples, sorted in descending order.

Theorems & Definitions (2)

• Definition 1: Generative Bias
• Definition 2: Minority Attribute