Unequal Opportunities: Examining the Bias in Geographical Recommendations by Large Language Models

TL;DR

This study audits Large Language Models (LLMs) for geographic biases in city recommendations across relocation, tourism, and business domains using Reddit-derived queries. It analyzes internal and external similarity among six LLMs with 24 prompts, employing metrics like Jaccard, TF-IDF, cosine similarity, and BLEU, alongside distributional measures (Concentration ratio, Theil index) and extrinsic comparisons to external city databases. The results show strong internal consistency in city lists but biased representations favoring affluent, highly educated areas and under-representing historically underserved groups, suggesting a rich-get-richer dynamic with real-world implications for mobility and opportunity. The authors discuss implications for designers, propose mitigations such as follow-up questions and diverse data integration, and call for more inclusive evaluation of LLM-enabled information services.

Abstract

Recent advancements in Large Language Models (LLMs) have made them a popular information-seeking tool among end users. However, the statistical training methods for LLMs have raised concerns about their representation of under-represented topics, potentially leading to biases that could influence real-world decisions and opportunities. These biases could have significant economic, social, and cultural impacts as LLMs become more prevalent, whether through direct interactions--such as when users engage with chatbots or automated assistants--or through their integration into third-party applications (as agents), where the models influence decision-making processes and functionalities behind the scenes. Our study examines the biases present in LLMs recommendations of U.S. cities and towns across three domains: relocation, tourism, and starting a business. We explore two key research questions: (i) How similar LLMs responses are, and (ii) How this similarity might favor areas with certain characteristics over others, introducing biases. We focus on the consistency of LLMs responses and their tendency to over-represent or under-represent specific locations. Our findings point to consistent demographic biases in these recommendations, which could perpetuate a ``rich-get-richer'' effect that widens existing economic disparities.

Figures (8)

• Figure 1: Internal comparison of single-constraint and generic conditions across LLMs using different similarity scores. $p$-value significance levels for the comparison between the two conditions are shown on the left side of each plot. Error bars reflect the variance of pair-wise scores comprising each distribution.
• Figure 2: External comparison of single-constraint and generic conditions across LLMs using different similarity scores. $p$-value significance levels for the comparison between the two conditions are shown on the left side of each plot. Error bars reflect the variance of pair-wise scores comprising each distribution.
• Figure 3: Theil Index and concentration ratio of elicited responses by LLM. Error bars reflect the variance of concentration ratio and Theil index across respective distributions.
• Figure 4: Skewness of attributes pertaining to historically underserved race and gender groups. Error bars describe the variance of an LLM distribution for a particular attribute.
• Figure 5: Skewness of attributes pertaining to unemployment, disability and poverty.Error bars describe the variance of an LLM distribution for a particular attribute.