Entangled in Representations: Mechanistic Investigation of Cultural Biases in Large Language Models

TL;DR

Entangled in Representations investigates how culture is encoded inside LLMs rather than only in outputs. It introduces Culturescope, a mechanistic interpretability workflow with three stages (inference, scoping-in, filtering) and a Cultural Flattening score that uses chi-square contributions $X_{k,y}$ and expected counts $E_{k,y}$ to quantify distortion across cultures. The findings show Western-dominance bias and culture flattening in internal representations, with low-resource cultures less biased due to limited knowledge, as evidenced by intrinsic CF analyses and extrinsic MCQ tests with hard negatives. The work offers a pathway to mitigating internal cultural biases and improving culturally aligned LLMs by increasing representational coverage and targeted data.

Abstract

The growing deployment of large language models (LLMs) across diverse cultural contexts necessitates a deeper understanding of LLMs' representations of different cultures. Prior work has focused on evaluating the cultural awareness of LLMs by only examining the text they generate. This approach overlooks the internal sources of cultural misrepresentation within the models themselves. To bridge this gap, we propose Culturescope, the first mechanistic interpretability-based method that probes the internal representations of different cultural knowledge in LLMs. We also introduce a cultural flattening score as a measure of the intrinsic cultural biases of the decoded knowledge from Culturescope. Additionally, we study how LLMs internalize cultural biases, which allows us to trace how cultural biases such as Western-dominance bias and cultural flattening emerge within LLMs. We find that low-resource cultures are less susceptible to cultural biases, likely due to the model's limited parametric knowledge. Our work provides a foundation for future research on mitigating cultural biases and enhancing LLMs' cultural understanding.

Figures (8)

• Figure 1: Given the question (about the popular domestic vacation spot in Greece), Culturescope first generates an answer to the cultural question at the Inference stage. Then, it reads the hidden representation from the Inference stage and elicits the cultural knowledge used for the answer ('Zakynthos') at the Scoping-in stage. We finalize a list of cultural knowledge after the Filtering stage. Culturescope unveils the internal mechanism of LLMs that cannot be revealed through the Inference stage alone.
• Figure 2: A cultural question about the popular family game in China from the BLEnD dataset myung2024blend. For the given question about China, if an LLM answers 'Yutnori', a popular family game in South Korea, it is caused by the effect of cultural flattening between South Korea and China. On the other hand, if the answer is 'Monopoly', the LLM is generating an answer from a high resource culture.
• Figure 3: We present the results from the CF score on BLEnD with Llama-3.1 in English. Percentages on the right reflect relative knowledge coverage and are defined as the ratio between the number of cultural knowledge generated for a given culture and the total number of generated concepts across all cultures.
• Figure 4: We present a heatmap of attention contribution scores (z-score normalized) for incorrect predictions of Llama-3.1. The x-axis groups correspond to the culture of the chosen option, while the y-axis groups correspond to the culture of the gold answer. For example, in Figure \ref{['fig:attention_map_heatmap']} (a), the High-Low cell (0.44) indicates the average attention contribution to option tokens from the High resource group when the gold answer belongs to the Low resource group.
• Figure 5: Prompt templates used for open-ended QA evaluations.