When Abundance Conceals Weakness: Knowledge Conflict in Multilingual Models

TL;DR

The paper tackles cross-lingual knowledge conflict in multilingual LLMs by introducing CLEAR, a framework that disentangles how language-conditioned parametric memories interact with multilingual external evidence across four tasks. It constructs multilingual datasets (ConflictQA-PopQA and ConflictQA-StrategyQA) spanning 10 languages and evaluates six LLMs, revealing a task-dependent dichotomy: high-resource languages drive resilience in reasoning tasks while linguistically aligned, lower-resource languages better correct entity-centric errors. The work uncovers an abundance-weakness paradox, where data scale supports one cognitive pathway but undercuts cross-lingual factual grounding in others. These insights highlight the need for truly multilingual robustness beyond English-centric benchmarks and resource-size proxies, with implications for retrieval-augmented and cross-lingual AI systems.

Abstract

Large Language Models (LLMs) encode vast world knowledge across multiple languages, yet their internal beliefs are often unevenly distributed across linguistic spaces. When external evidence contradicts these language-dependent memories, models encounter \emph{cross-lingual knowledge conflict}, a phenomenon largely unexplored beyond English-centric settings. We introduce \textbf{CLEAR}, a \textbf{C}ross-\textbf{L}ingual knowl\textbf{E}dge conflict ev\textbf{A}luation f\textbf{R}amework that systematically examines how multilingual LLMs reconcile conflicting internal beliefs and multilingual external evidence. CLEAR decomposes conflict resolution into four progressive scenarios, from multilingual parametric elicitation to competitive multi-source cross-lingual induction, and systematically evaluates model behavior across two complementary QA benchmarks with distinct task characteristics. We construct multilingual versions of ConflictQA and ConflictingQA covering 10 typologically diverse languages and evaluate six representative LLMs. Our experiments reveal a task-dependent decision dichotomy. In reasoning-intensive tasks, conflict resolution is dominated by language resource abundance, with high-resource languages exerting stronger persuasive power. In contrast, for entity-centric factual conflicts, linguistic affinity, not resource scale, becomes decisive, allowing low-resource but linguistically aligned languages to outperform distant high-resource ones.

Figures (7)

• Figure 1: language distribution in the CLEAR framework: languages are mapped based on their resource richness and taxonomic family, enabling a systematic study of how linguistic affinity and data scale influence cross-lingual knowledge conflict resolution.
• Figure 2: Overview of the CLEAR framework.
• Figure 3: Task-dependent Persuasion Rate difference.$\Delta$ denotes the difference in Persuasion Rate between PopQA and StrategyQA (PopQA $-$ StrategyQA), highlighting task-specific induction behavior.
• Figure 4: Aggregate accuracy in Multi-Source Conflict Resolution. Accuracy averaged over six models, stratified by query language and evaluation benchmark.
• Figure 5: Task 3 results: Persuasion Rate and Stubborn Rate on PopQA and StrategyQA across four representative models.