Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MPN: Leveraging Multilingual Patch Neuron for Cross-lingual Model Editing

Nianwen Si, Hao Zhang, Weiqiang Zhang

TL;DR

This work tackles the problem of keeping multilingual LLMs up to date by addressing cross-lingual editing, where updates in one language do not reliably propagate to others. It introduces Multilingual Patch Neuron (MPN), a simple input-side augmentation to Transformer-patcher-based editing that trains patch neurons using English editing data in conjunction with parallel corpora, enabling cross-lingual updates with minimal changes to existing methods. Empirical results on XFEVER and XNLI show that MPN substantially improves cross-lingual generalization (CLG) by about 9–12% on XFEVER and around 14% on XNLI, while preserving locality and yielding good monolingual generalization. The method leverages the multilingual model’s inherent cross-lingual representations by embedding cross-lingual sampling into the editing process, offering a practical, easily adaptable path to multilingual model editing.

Abstract

Large language models are known for encoding a vast amount of factual knowledge, but they often becomes outdated due to the ever-changing nature of external information. A promising solution to this challenge is the utilization of model editing methods to update the knowledge in an efficient manner. However, the majority of existing model editing techniques are limited to monolingual frameworks, thus failing to address the crucial issue of cross-lingual knowledge synchronization for multilingual models. To tackle this problem, we propose a simple yet effective method that trains multilingual patch neuron to store cross-lingual knowledge. It can be easily adapted to existing approaches to enhance their cross-lingual editing capabilities. To evaluate our method, we conduct experiments using both the XNLI dataset and a self-constructed XFEVER dataset. Experimental results demonstrate that our proposed method achieves improved performance in cross-lingual editing tasks without requiring excessive modifications to the original methodology, thereby showcasing its user-friendly characteristics. Codes will be released soon.

MPN: Leveraging Multilingual Patch Neuron for Cross-lingual Model Editing

TL;DR

This work tackles the problem of keeping multilingual LLMs up to date by addressing cross-lingual editing, where updates in one language do not reliably propagate to others. It introduces Multilingual Patch Neuron (MPN), a simple input-side augmentation to Transformer-patcher-based editing that trains patch neurons using English editing data in conjunction with parallel corpora, enabling cross-lingual updates with minimal changes to existing methods. Empirical results on XFEVER and XNLI show that MPN substantially improves cross-lingual generalization (CLG) by about 9–12% on XFEVER and around 14% on XNLI, while preserving locality and yielding good monolingual generalization. The method leverages the multilingual model’s inherent cross-lingual representations by embedding cross-lingual sampling into the editing process, offering a practical, easily adaptable path to multilingual model editing.

Abstract

Large language models are known for encoding a vast amount of factual knowledge, but they often becomes outdated due to the ever-changing nature of external information. A promising solution to this challenge is the utilization of model editing methods to update the knowledge in an efficient manner. However, the majority of existing model editing techniques are limited to monolingual frameworks, thus failing to address the crucial issue of cross-lingual knowledge synchronization for multilingual models. To tackle this problem, we propose a simple yet effective method that trains multilingual patch neuron to store cross-lingual knowledge. It can be easily adapted to existing approaches to enhance their cross-lingual editing capabilities. To evaluate our method, we conduct experiments using both the XNLI dataset and a self-constructed XFEVER dataset. Experimental results demonstrate that our proposed method achieves improved performance in cross-lingual editing tasks without requiring excessive modifications to the original methodology, thereby showcasing its user-friendly characteristics. Codes will be released soon.
Paper Structure (20 sections, 13 equations, 3 figures, 4 tables)

This paper contains 20 sections, 13 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Model Editing
  4. Cross-lingual Transfer
  5. Cross-lingual Model Editing
  6. Task Definition
  7. Monolingual Baseline
  8. Multilingual Patch Neuron
  9. Experiments
  10. Settings
  11. Models and Datasets
  12. Evaluation Metrics
  13. Overall Results
  14. Results on XFEVER
  15. Results on XNLI
  16. ...and 5 more sections

Figures (3)

  • Figure 1: Monolingual model editing fails in cross-lingual scenario
  • Figure 2: The patch neuron in FFN layer
  • Figure 3: The distribution of sentence length for six languages on XFEVER dataset