Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sima AIunty: Caste Audit in LLM-Driven Matchmaking

Atharva Naik, Shounok Kar, Varnika Sharma, Ashwin Rajadesingan, Koustuv Saha

Abstract

Social and personal decisions in relational domains such as matchmaking are deeply entwined with cultural norms and historical hierarchies, and can potentially be shaped by algorithmic and AI-mediated assessments of compatibility, acceptance, and stability. In South Asian contexts, caste remains a central aspect of marital decision-making, yet little is known about how contemporary large language models (LLMs) reproduce or disrupt caste-based stratification in such settings. In this work, we conduct a controlled audit of caste bias in LLM-mediated matchmaking evaluations using real-world matrimonial profiles. We vary caste identity across Brahmin, Kshatriya, Vaishya, Shudra, and Dalit, and income across five buckets, and evaluate five LLM families (GPT, Gemini, Llama, Qwen, and BharatGPT). Models are prompted to assess profiles along dimensions of social acceptance, marital stability, and cultural compatibility. Our analysis reveals consistent hierarchical patterns across models: same-caste matches are rated most favorably, with average ratings up to 25% higher (on a 10-point scale) than inter-caste matches, which are further ordered according to traditional caste hierarchy. These findings highlight how existing caste hierarchies are reproduced in LLM decision-making and underscore the need for culturally grounded evaluation and intervention strategies in AI systems deployed in socially sensitive domains, where such systems risk reinforcing historical forms of exclusion.

Sima AIunty: Caste Audit in LLM-Driven Matchmaking

Abstract

Social and personal decisions in relational domains such as matchmaking are deeply entwined with cultural norms and historical hierarchies, and can potentially be shaped by algorithmic and AI-mediated assessments of compatibility, acceptance, and stability. In South Asian contexts, caste remains a central aspect of marital decision-making, yet little is known about how contemporary large language models (LLMs) reproduce or disrupt caste-based stratification in such settings. In this work, we conduct a controlled audit of caste bias in LLM-mediated matchmaking evaluations using real-world matrimonial profiles. We vary caste identity across Brahmin, Kshatriya, Vaishya, Shudra, and Dalit, and income across five buckets, and evaluate five LLM families (GPT, Gemini, Llama, Qwen, and BharatGPT). Models are prompted to assess profiles along dimensions of social acceptance, marital stability, and cultural compatibility. Our analysis reveals consistent hierarchical patterns across models: same-caste matches are rated most favorably, with average ratings up to 25% higher (on a 10-point scale) than inter-caste matches, which are further ordered according to traditional caste hierarchy. These findings highlight how existing caste hierarchies are reproduced in LLM decision-making and underscore the need for culturally grounded evaluation and intervention strategies in AI systems deployed in socially sensitive domains, where such systems risk reinforcing historical forms of exclusion.

Paper Structure

This paper contains 27 sections, 2 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Society and Caste
  4. Caste and Matchmaking
  5. Caste in Technology and Design
  6. Auditing Bias and Harms in AI Systems
  7. Data and Methods
  8. Seed Dataset
  9. Simulated Data Transformations
  10. LLM-Generated Matchmaking Evaluations
  11. Regression Analysis to Examine Caste Bias
  12. Results
  13. LLM Evaluations by Caste
  14. Regression Modeling of Evaluation Outcomes
  15. Social Acceptance
  16. ...and 12 more sections

Figures (2)

  • Figure 1: Heatmaps of the average ratings provided to a male evaluator with female candidates across caste groups of Brahmin (B), Kshatriya (K), Vaishya (V), Shudra (S), and Dalit (D).
  • Figure 2: Heatmaps of the average ratings provided to a female evaluator with male candidates across caste groups of Brahmin (B), Kshatriya (K), Vaishya (V), Shudra (S), and Dalit (D).