Model Editing at Scale leads to Gradual and Catastrophic Forgetting

TL;DR

This work investigates the scalability of knowledge editing in large language models by evaluating ROME and MEMIT under thousands of sequential edits. It uncovers two forgetting phases—gradual and catastrophic—and introduces disabling edits as a fundamental risk, showing that edits bleed into unrelated facts and degrade downstream tasks. The study argues that scalable model editing requires robust evaluation beyond edit-success metrics and presents MEMIT as relatively more robust than ROME, though both face severe forgetting at scale. The findings highlight critical limitations in current approaches and call for new methods that preserve prior edits and downstream capabilities while enabling large-scale updates.

Abstract

Editing knowledge in large language models is an attractive capability to have which allows us to correct incorrectly learnt facts during pre-training, as well as update the model with an ever-growing list of new facts. While existing model editing techniques have shown promise, they are usually evaluated using metrics for reliability, specificity and generalization over one or few edits. We argue that for model editing to have practical utility, we must be able to make multiple edits to the same model. With this in mind, we evaluate the current model editing methods at scale, focusing on two state of the art methods: ROME and MEMIT. We find that as the model is edited sequentially with multiple facts, it continually forgets previously edited facts and the ability to perform downstream tasks. This forgetting happens in two phases -- an initial gradual but progressive forgetting phase followed by abrupt or catastrophic forgetting phase. Both gradual and catastrophic forgetting limit the usefulness of model editing methods at scale -- the former making model editing less effective as multiple edits are made to the model while the latter caps the scalability of such model editing methods. Our analysis also highlights other key limitations of ROME and MEMIT at scale. With our work, we push for the development and evaluation of model editing methods keeping scalability in mind.

Figures (36)

• Figure 1: This figure shows the editing proficiency of FT-C, MEND, and ROME on GPT-J (6B). The dotted line represents the metric averaged over a past window size of 5, whereas the solid lines represent the metric averaged over a past window size of 50. Figure \ref{['fig:forgetting']} show the percentage of previously edited facts forgotten as a function of number of edits.
• Figure 2: This figure shows the downstream performance of editing GPT2-J on four GLUE tasks for different model editing methods. Figure \ref{['fig:distance']} shows the the normalized distance between the edited layer and its original weights.
• Figure 3: This figure shows the editing proficiency of MEMIT on GPT-J for Sample 1 over 2000 sequential edits made to the model.
• Figure 4: Compares the forgetting rate between ROME and MEMIT.
• Figure 5: Editing proficiency plots for Sample 1 for GPT-XL (1.5B).