One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Models

TL;DR

A novel method named Multilingual Integrated Gradients with Uncertainty Estimation (MATRICE), which quantifies the uncertainty across queries and languages during knowledge localization and can accurately localize language-agnostic knowledge neurons, is proposed.

Abstract

Large language models (LLMs) have learned vast amounts of factual knowledge through self-supervised pre-training on large-scale corpora. Meanwhile, LLMs have also demonstrated excellent multilingual capabilities, which can express the learned knowledge in multiple languages. However, the knowledge storage mechanism in LLMs still remains mysterious. Some researchers attempt to demystify the factual knowledge in LLMs from the perspective of knowledge neurons, and subsequently discover language-agnostic knowledge neurons that store factual knowledge in a form that transcends language barriers. However, the preliminary finding suffers from two limitations: 1) High Uncertainty in Localization Results. Existing study only uses a prompt-based probe to localize knowledge neurons for each fact, while LLMs cannot provide consistent answers for semantically equivalent queries. Thus, it leads to inaccurate localization results with high uncertainty. 2) Lack of Analysis in More Languages. The study only analyzes language-agnostic knowledge neurons on English and Chinese data, without exploring more language families and languages. Naturally, it limits the generalizability of the findings. To address aforementioned problems, we first construct a new benchmark called Rephrased Multilingual LAMA (RML-LAMA), which contains high-quality cloze-style multilingual parallel queries for each fact. Then, we propose a novel method named Multilingual Integrated Gradients with Uncertainty Estimation (MATRICE), which quantifies the uncertainty across queries and languages during knowledge localization. Extensive experiments show that our method can accurately localize language-agnostic knowledge neurons. We also further investigate the role of language-agnostic knowledge neurons in cross-lingual knowledge editing, knowledge enhancement and new knowledge injection.

Figures (7)

• Figure 1: An example of LLMs answering multilingual queries correctly. Language-agnostic knowledge neurons are simultaneously activated by queries from multiple languages.
• Figure 2: The process of obtaining language-agnostic knowledge neurons (LAKNs) using the previous method chen2024journey: intersection of knowledge neurons (KNs) in Chinese and English.
• Figure 3: The architecture of our proposed multilingual integrated gradients with uncertainty estimation (MaTrice) for calculating the language-agnostic attribution score for each neuron. "Query 1", "Query 2" and "Query T" denote semantically equivalent queries for one fact. "Score 1" is the attribution score for each neuron, when using "Query 1" for knowledge attribution. "LAA score" denotes the language-agnostic attribution score.
• Figure 4: The distribution of language-agnostic knowledge neurons in four multilingual LLMs, including mGPT, LLaMA2, LLaMA3 and mBERT.
• Figure 5: Results of suppressing or enhancing language-agnostic knowledge neurons experiment. The "Probability Change Rate" denotes the probability change ratio of correct answers, which is computed via Equation (\ref{['E14']}).