Meta-Cultural Competence: Climbing the Right Hill of Cultural Awareness
Sougata Saha, Saurabh Kumar Pandey, Monojit Choudhury
TL;DR
The paper argues that large language models exhibit Western-centric biases and that fixing this requires meta-cultural competence rather than mere cultural awareness. It introduces the Multi-Pair Octopus Test to motivate continual, adaptive handling of unseen cultures and formulates two core competencies—Variational Awareness and Explication/Negotiation—to measure and guide system development. A formal entropy-based Delta metric is proposed to quantify variational awareness via $f_v(C)$ and $\\hat{f}_v(C)$, with an illustrative experiment on driving norms and GeoMLAMA showing substantial cross-cultural variability and a weak link between awareness and accuracy. The work outlines open questions and advocates for interdisciplinary efforts to design, train, evaluate, and deploy meta-culturally competent AI across diverse, dynamic cultural landscapes.
Abstract
Numerous recent studies have shown that Large Language Models (LLMs) are biased towards a Western and Anglo-centric worldview, which compromises their usefulness in non-Western cultural settings. However, "culture" is a complex, multifaceted topic, and its awareness, representation, and modeling in LLMs and LLM-based applications can be defined and measured in numerous ways. In this position paper, we ask what does it mean for an LLM to possess "cultural awareness", and through a thought experiment, which is an extension of the Octopus test proposed by Bender and Koller (2020), we argue that it is not cultural awareness or knowledge, rather meta-cultural competence, which is required of an LLM and LLM-based AI system that will make it useful across various, including completely unseen, cultures. We lay out the principles of meta-cultural competence AI systems, and discuss ways to measure and model those.