Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Large Language Models Naively Recover Ethnicity from Individual Records

Noah Dasanaike

TL;DR

The paper demonstrates that large language models can accurately infer ethnicity-related categories from names across diverse contexts, often outperforming BISG and extending beyond US-centric categories to religion, caste, and other identities. By using simple prompts that combine names with geography and optional metadata, the method achieves high accuracy in US samples and meaningful cross-national validation across Lebanon, India, and six additional countries, without requiring labeled training data. It further shows that small transformer models fine-tuned with LoRA on LLM-generated labels can match or approach teacher performance while enabling local deployment and reducing API dependence. The findings suggest LLM-based name-ethnicity inference is a flexible, scalable alternative to BISG for social science research, with careful attention to context-specific naming conventions and potential biases, privacy, and reproducibility considerations.

Abstract

I demonstrate that large language models can infer ethnicity from names with accuracy exceeding that of Bayesian Improved Surname Geocoding (BISG) without additional training data, enabling inference outside the United States and to contextually appropriate classification categories. Using stratified samples from Florida and North Carolina voter files with self-reported race, LLM-based classification achieves up to 84.7% accuracy, outperforming BISG (68.2%) on balanced samples. I test six models including Gemini 3 Flash, GPT-4o, and open-source alternatives such as DeepSeek v3.2 and GLM-4.7. Enabling extended reasoning can improve accuracy by 1-3 percentage points, though effects vary across contexts; including metadata such as party registration reaches 86.7%. LLM classification also reduces the income bias inherent in BISG, where minorities in wealthier neighborhoods are systematically misclassified as White. I further validate using Lebanese voter registration with religious sect (64.3% accuracy), Indian MPs from reserved constituencies (99.2%), and Indian land records with caste classification (74.0%). Aggregate validation across India, Uganda, Nepal, Armenia, Chile, and Costa Rica using original full-count voter rolls demonstrates that the method recovers known population distributions where naming conventions are distinctive. For large-scale applications, small transformer models fine-tuned on LLM labels exceed BISG accuracy while enabling local deployment at no cost.

Large Language Models Naively Recover Ethnicity from Individual Records

TL;DR

The paper demonstrates that large language models can accurately infer ethnicity-related categories from names across diverse contexts, often outperforming BISG and extending beyond US-centric categories to religion, caste, and other identities. By using simple prompts that combine names with geography and optional metadata, the method achieves high accuracy in US samples and meaningful cross-national validation across Lebanon, India, and six additional countries, without requiring labeled training data. It further shows that small transformer models fine-tuned with LoRA on LLM-generated labels can match or approach teacher performance while enabling local deployment and reducing API dependence. The findings suggest LLM-based name-ethnicity inference is a flexible, scalable alternative to BISG for social science research, with careful attention to context-specific naming conventions and potential biases, privacy, and reproducibility considerations.

Abstract

I demonstrate that large language models can infer ethnicity from names with accuracy exceeding that of Bayesian Improved Surname Geocoding (BISG) without additional training data, enabling inference outside the United States and to contextually appropriate classification categories. Using stratified samples from Florida and North Carolina voter files with self-reported race, LLM-based classification achieves up to 84.7% accuracy, outperforming BISG (68.2%) on balanced samples. I test six models including Gemini 3 Flash, GPT-4o, and open-source alternatives such as DeepSeek v3.2 and GLM-4.7. Enabling extended reasoning can improve accuracy by 1-3 percentage points, though effects vary across contexts; including metadata such as party registration reaches 86.7%. LLM classification also reduces the income bias inherent in BISG, where minorities in wealthier neighborhoods are systematically misclassified as White. I further validate using Lebanese voter registration with religious sect (64.3% accuracy), Indian MPs from reserved constituencies (99.2%), and Indian land records with caste classification (74.0%). Aggregate validation across India, Uganda, Nepal, Armenia, Chile, and Costa Rica using original full-count voter rolls demonstrates that the method recovers known population distributions where naming conventions are distinctive. For large-scale applications, small transformer models fine-tuned on LLM labels exceed BISG accuracy while enabling local deployment at no cost.
Paper Structure (8 sections, 1 figure, 10 tables)

This paper contains 8 sections, 1 figure, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Results
  5. Scaling
  6. Reasoning
  7. Income Bias
  8. Discussion

Figures (1)

  • Figure 1: Misclassification rate by ZCTA median household income (ACS 5-year estimates) and race, North Carolina voters. Points show average misclassification rate for each income ventile (5% quantile), sized by number of observations. Lines show loess fits. For Black voters, BISG misclassification increases sharply with neighborhood income; LLM shows no such gradient.