Edit Once, Update Everywhere: A Simple Framework for Cross-Lingual Knowledge Synchronization in LLMs

TL;DR

This paper introduces X-KDE, a cross-lingual knowledge editing framework that propagates updates from a source language to target languages. It comprises two stages: XE-IT, which fine-tunes on a curated parallel dataset to modify in-scope knowledge while preserving unrelated content, and TL-PO, which uses ORPO-based alignment to ensure outputs in the target language reflect the updates. A high-quality cross-lingual dataset is introduced to bolster transfer across languages. Empirical results on Bi-ZsRE and MzsRE demonstrate cross-lingual gains and robustness across monolingual tasks, establishing X-KDE as a new SOTA approach for cross-lingual knowledge editing with scalable batch and sequential editing capabilities and strong generalization across languages.

Abstract

Knowledge editing allows for efficient adaptation of large language models (LLMs) to new information or corrections without requiring full retraining. However, prior methods typically focus on either single-language editing or basic multilingual editing, failing to achieve true cross-linguistic knowledge synchronization. To address this, we present a simple and practical state-of-the-art (SOTA) recipe Cross-Lingual Knowledge Democracy Edit (X-KDE), designed to propagate knowledge from a dominant language to other languages effectively. Our X-KDE comprises two stages: (i) Cross-lingual Edition Instruction Tuning (XE-IT), which fine-tunes the model on a curated parallel dataset to modify in-scope knowledge while preserving unrelated information, and (ii) Target-language Preference Optimization (TL-PO), which applies advanced optimization techniques to ensure consistency across languages, fostering the transfer of updates. Additionally, we contribute a high-quality, cross-lingual dataset, specifically designed to enhance knowledge transfer across languages. Extensive experiments on the Bi-ZsRE and MzsRE benchmarks show that X-KDE significantly enhances cross-lingual performance, achieving an average improvement of +8.19%, while maintaining high accuracy in monolingual settings.

Figures (4)

• Figure 1: Examples of (a) monolingual and (b) cross-lingual knowledge editing. In the former, the editing and verification languages are the same, while in the latter, knowledge is transferred from the source language (e.g., English) to the target language (e.g., Chinese).
• Figure 2: Illustration of Cross-Lingual Knowledge Democracy Edit (X-KDE) framework. In the XE-IT phase, we fine-tune the LLM on a carefully curated parallel dataset, enabling it to incorporate newly edited information from the source language when queried in the target language. In the TL-PO phase, multiple responses are generated, ranked based on similarity to the target language answer, and alignment optimization is applied to refine the output.
• Figure 3: Mean batch-editing performance across four benchmarks at batch sizes 1, 10, 100, and 1000.
• Figure 4: Mean sequential-editing results across four knowledge editing benchmarks, shown for data stream sizes of 1, 10, 100, 500, and 1000 (log-scale).