Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Equilibrium Dynamics and Mitigation of Gender Bias in Synthetically Generated Data

Ashish Kattamuri, Arpita Vats, Harshwardhan Fartale, Rahul Raja, Akshata Kishore Moharir, Ishita Prasad

TL;DR

This work investigates gender bias dynamics in recursive synthetic data generation with large language models, revealing equilibrium rather than universal amplification. It employs three bias metrics—rule-based, embedding-based, and downstream—across three generations and compares four mitigation strategies, including contrastive augmentation. A key finding is that the model stabilizes around an intrinsic bias level near $0.11$–$0.13$, with low seed bias amplifying to +$36\%$ and high seed bias decaying by $-26\%$, while contrastive augmentation delivers about $91\%$ average reduction in downstream bias despite higher embedding bias. The results underscore the importance of multidimensional fairness evaluation and model-specific mitigation for responsible synthetic data generation in practical AI systems.

Abstract

Recursive prompting with large language models enables scalable synthetic dataset generation but introduces the risk of bias amplification. We investigate gender bias dynamics across three generations of recursive text generation using three complementary evaluation frameworks: rule-based pattern matching, embedding-based semantic similarity, and downstream task performance. Experiments with three initial bias levels (0.1, 0.3, 0.6) and four mitigation strategies reveal equilibrium dynamics rather than monotonic amplification. The low initial bias amplifies toward the model's inherent bias level (+36%), whereas the high initial bias decays toward it (-26%). Among mitigation methods, contrastive augmentation, which introduces gender-swapped variants, achieves significant downstream bias reduction (98.8% for low initial bias and 91% on average) despite producing higher embedding-based bias scores. This paradox demonstrates that semantic similarity metrics may diverge from behavioral fairness outcomes, highlighting the need for multidimensional evaluation in responsible synthetic data generation.

Equilibrium Dynamics and Mitigation of Gender Bias in Synthetically Generated Data

TL;DR

This work investigates gender bias dynamics in recursive synthetic data generation with large language models, revealing equilibrium rather than universal amplification. It employs three bias metrics—rule-based, embedding-based, and downstream—across three generations and compares four mitigation strategies, including contrastive augmentation. A key finding is that the model stabilizes around an intrinsic bias level near –, with low seed bias amplifying to + and high seed bias decaying by , while contrastive augmentation delivers about average reduction in downstream bias despite higher embedding bias. The results underscore the importance of multidimensional fairness evaluation and model-specific mitigation for responsible synthetic data generation in practical AI systems.

Abstract

Recursive prompting with large language models enables scalable synthetic dataset generation but introduces the risk of bias amplification. We investigate gender bias dynamics across three generations of recursive text generation using three complementary evaluation frameworks: rule-based pattern matching, embedding-based semantic similarity, and downstream task performance. Experiments with three initial bias levels (0.1, 0.3, 0.6) and four mitigation strategies reveal equilibrium dynamics rather than monotonic amplification. The low initial bias amplifies toward the model's inherent bias level (+36%), whereas the high initial bias decays toward it (-26%). Among mitigation methods, contrastive augmentation, which introduces gender-swapped variants, achieves significant downstream bias reduction (98.8% for low initial bias and 91% on average) despite producing higher embedding-based bias scores. This paradox demonstrates that semantic similarity metrics may diverge from behavioral fairness outcomes, highlighting the need for multidimensional evaluation in responsible synthetic data generation.

Paper Structure

This paper contains 9 sections, 3 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Bias Measurement
  4. Results
  5. Equilibrium Dynamics in Vanilla Condition
  6. Mitigation Strategy Comparison
  7. Statistical Analysis
  8. Discussion
  9. Conclusion

Figures (3)

  • Figure 1: Embedding bias evolution across three generations for initial bias levels 0.1 (left), 0.3 (center), and 0.6 (right). Vanilla (blue) demonstrates equilibrium dynamics, where low bias amplifies and high bias decays. Contrastive (orange) yields higher embedding bias but lower downstream bias.
  • Figure 2: Effect sizes (Gen-3 minus Gen-0) for embedding bias across strategies. Negative values indicate bias decay. Vanilla shows the strongest decay, while contrastive augmentation achieves the best downstream fairness.
  • Figure 3: Gen-3 embedding bias across strategies and initial bias levels. Darker shades represent higher embedding bias. Despite these higher values, contrastive augmentation achieves the lowest downstream bias.