Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mitigating Bias in Queer Representation within Large Language Models: A Collaborative Agent Approach

Tianyi Huang, Arya Somasundaram

TL;DR

This work tackles pronoun bias in large language models by introducing a collaborative multi-agent pipeline that detects and optimizes pronoun usage for queer inclusivity. The three-Agent framework—Assistant, Language Analysis, and Optimization—offers transparent, reasoned decisions about pronoun inclusivity and is evaluated on the Tango Dataset, showing significant improvements over GPT-4o for traditionally gendered pronouns and strong performance on non-binary pronouns. Key contributions include a strict JSON-output protocol, rigorous statistical validation (e.g., $\chi^2$ tests with $p$-values), and a demonstration that agent-driven bias mitigation can enhance fairness in AI-generated content. The findings support the viability of agent-based approaches for inclusive AI and point to future work integrating contextual reasoning and retrieval-augmented generation for broader applicability.

Abstract

Large Language Models (LLMs) often perpetuate biases in pronoun usage, leading to misrepresentation or exclusion of queer individuals. This paper addresses the specific problem of biased pronoun usage in LLM outputs, particularly the inappropriate use of traditionally gendered pronouns ("he," "she") when inclusive language is needed to accurately represent all identities. We introduce a collaborative agent pipeline designed to mitigate these biases by analyzing and optimizing pronoun usage for inclusivity. Our multi-agent framework includes specialized agents for both bias detection and correction. Experimental evaluations using the Tango dataset-a benchmark focused on gender pronoun usage-demonstrate that our approach significantly improves inclusive pronoun classification, achieving a 32.6 percentage point increase over GPT-4o in correctly disagreeing with inappropriate traditionally gendered pronouns $(χ^2 = 38.57, p < 0.0001)$. These results accentuate the potential of agent-driven frameworks in enhancing fairness and inclusivity in AI-generated content, demonstrating their efficacy in reducing biases and promoting socially responsible AI.

Mitigating Bias in Queer Representation within Large Language Models: A Collaborative Agent Approach

TL;DR

This work tackles pronoun bias in large language models by introducing a collaborative multi-agent pipeline that detects and optimizes pronoun usage for queer inclusivity. The three-Agent framework—Assistant, Language Analysis, and Optimization—offers transparent, reasoned decisions about pronoun inclusivity and is evaluated on the Tango Dataset, showing significant improvements over GPT-4o for traditionally gendered pronouns and strong performance on non-binary pronouns. Key contributions include a strict JSON-output protocol, rigorous statistical validation (e.g., tests with -values), and a demonstration that agent-driven bias mitigation can enhance fairness in AI-generated content. The findings support the viability of agent-based approaches for inclusive AI and point to future work integrating contextual reasoning and retrieval-augmented generation for broader applicability.

Abstract

Large Language Models (LLMs) often perpetuate biases in pronoun usage, leading to misrepresentation or exclusion of queer individuals. This paper addresses the specific problem of biased pronoun usage in LLM outputs, particularly the inappropriate use of traditionally gendered pronouns ("he," "she") when inclusive language is needed to accurately represent all identities. We introduce a collaborative agent pipeline designed to mitigate these biases by analyzing and optimizing pronoun usage for inclusivity. Our multi-agent framework includes specialized agents for both bias detection and correction. Experimental evaluations using the Tango dataset-a benchmark focused on gender pronoun usage-demonstrate that our approach significantly improves inclusive pronoun classification, achieving a 32.6 percentage point increase over GPT-4o in correctly disagreeing with inappropriate traditionally gendered pronouns . These results accentuate the potential of agent-driven frameworks in enhancing fairness and inclusivity in AI-generated content, demonstrating their efficacy in reducing biases and promoting socially responsible AI.

Paper Structure

This paper contains 16 sections, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Agents and their Roles
  5. Design Considerations
  6. Implementation Details
  7. Results
  8. Findings
  9. Statistical Significance
  10. Implications
  11. Ethical Considerations
  12. Conclusion
  13. Limitations and Future Work
  14. Appendix
  15. Supplementary Resources
  16. ...and 1 more sections

Figures (2)

  • Figure 1: This diagram illustrates a multi-agent workflow designed to enhance language inclusivity in user queries. The process begins with the Assistant Agent, which analyzes the query for gender-neutral language and provides an initial inclusivity assessment. Next, the Language Analysis Agent reviews and refines this assessment with critical reasoning. Finally, the Optimizer Agent validates and finalizes the decision on pronoun inclusivity, producing a Boolean result and detailed reasoning. This structured approach ensures a thorough inclusivity evaluation.
  • Figure 2: Example decision-making process of the Agent Workflow, showing both agreement and disagreement cases handled by the Assistant Agent, Language Analysis Agent, and Optimizer Agent.