The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapse

TL;DR

This work reveals that even a single fact edit can trigger collapse in large language models, challenging the practicality of current model-editing approaches. By introducing ME-PPL, a perplexity-based surrogate, and two challenging benchmarks (HardCF and HardEdit), the authors demonstrate that most editing methods induce notable degradation in downstream tasks, especially under sequential edits. The study highlights significant risks in deploying editing techniques in real-world systems and calls for robust editing methods and stress-testing benchmarks to ensure reliability. The findings establish a framework for rapid collapse detection and provide a dataset to guide future research toward safer, more dependable model editing.

Abstract

Although model editing has shown promise in revising knowledge in Large Language Models (LLMs), its impact on the inherent capabilities of LLMs is often overlooked. In this work, we reveal a critical phenomenon: even a single edit can trigger model collapse, manifesting as significant performance degradation in various benchmark tasks. However, benchmarking LLMs after each edit, while necessary to prevent such collapses, is impractically time-consuming and resource-intensive. To mitigate this, we propose using perplexity as a surrogate metric, validated by extensive experiments demonstrating changes in an edited model's perplexity are strongly correlated with its downstream task performances. We further conduct an in-depth study on sequential editing, a practical setting for real-world scenarios, across various editing methods and LLMs, focusing on hard cases from our previous single edit studies. The results indicate that nearly all examined editing methods result in model collapse after only few edits. To facilitate further research, we have utilized GPT-3.5 to develop a new dataset, HardEdit, based on those hard cases. This dataset aims to establish the foundation for pioneering research in reliable model editing and the mechanisms underlying editing-induced model collapse. We hope this work can draw the community's attention to the potential risks inherent in model editing practices.

Figures (15)

• Figure 1: (a) Editing GPT-J with ROME to inject a new fact "Twitter was acquired by Elon Musk" severely disrupts its ability to generate coherent text. (b) The downstream tasks performance of the edited GPT-J in Figure \ref{['fig:collapse']} has significantly deteriorated, approaching the "random" baseline indicative of mere guesswork.
• Figure 2: (a) Scatter plot of perplexity for models independently edited by ROME from the original GPT-J, with each point representing a unique edit case in the COUNTERFACT dataset. "Case ID" refers to the index of each edit sample. (b) Average performance with variance on downstream tasks for the top 30 high-perplexity models in Figure \ref{['fig:rome_gptj_ppl']}, comparing to the original model and random guessing.
• Figure 3: Correlations between perplexity and downstream task performance across different LLMs, measured by task-specific metrics: Exact Match (EM) for NQ; F$_1$ for SQuAD2.0.; Accuracy for remaining tasks. $\rho$ refers to the Spearman's Rho value, measuring the rank correlation between perplexity and corresponding downstream task performance, with all $p$-values $<$ 0.01.
• Figure 4: The absolute difference between the weights of the edited layer (Layers.5.mlp.down_proj) and its original weights for ROME-edited Llama2-7b models.
• Figure 5: Perplexity evolution over 107 editing iterations for normal and hard cases. The y-axes are tailored for each subplot accordingly due to the the significant variation in the magnitude of perplexity changes.