Behemoth: Benchmarking Unlearning in LLMs Using Fully Synthetic Data

TL;DR

Behemoth introduces a fully synthetic data framework to study how LLMs store and edit factual knowledge, enabling precise measurement of knowledge edits divorced from real-world data. It trains a $3.1\times10^{7}$-parameter GPT-style Transformer on deliberately constructed {subject, relation, object} facts and evaluates editing methods (full finetuning, LoRA, and ROMEmeng2022locating) across independent, correlated, and nested data regimes. The findings show that targeted, low-rank updates often suffice for simple edits, but robust forgetting and more complex dependencies may require higher-rank or full updates, while interpretability tools offer limited guidance for layer choice. The Behemoth framework thus provides a scalable, controllable platform for benchmarking model editing and understanding data-storage interactions, with potential extensions to richer grammars and other model properties in future work.

Abstract

As artificial neural networks, and specifically large language models, have improved rapidly in capabilities and quality, they have increasingly been deployed in real-world applications, from customer service to Google search, despite the fact that they frequently make factually incorrect or undesirable statements. This trend has inspired practical and academic interest in model editing, that is, in adjusting the weights of the model to modify its likely outputs for queries relating to a specific fact or set of facts. This may be done either to amend a fact or set of facts, for instance, to fix a frequent error in the training data, or to suppress a fact or set of facts entirely, for instance, in case of dangerous knowledge. Multiple methods have been proposed to do such edits. However, at the same time, it has been shown that such model editing can be brittle and incomplete. Moreover the effectiveness of any model editing method necessarily depends on the data on which the model is trained, and, therefore, a good understanding of the interaction of the training data distribution and the way it is stored in the network is necessary and helpful to reliably perform model editing. However, working with large language models trained on real-world data does not allow us to understand this relationship or fully measure the effects of model editing. We therefore propose Behemoth, a fully synthetic data generation framework. To demonstrate the practical insights from the framework, we explore model editing in the context of simple tabular data, demonstrating surprising findings that, in some cases, echo real-world results, for instance, that in some cases restricting the update rank results in a more effective update. The code is available at https://github.com/IST-DASLab/behemoth.git.

Figures (27)

• Figure 1: Success of ROME, full, and LoRA finetuning for the simple dataset scenario. All results are averaged across three runs.
• Figure 2: Success of ROME, full, and LoRA finetuning for the correlated relationship dataset scenario. All results are averaged across three runs.
• Figure 3: Success of ROME, full, and LoRA finetuning for the nested relationship dataset scenario. All results are averaged across three runs.
• Figure 4: Simple dataset. Ability to effect the change (top) while preserving the rest of the model accuracy (bottom) of, from left to right, making a single override, ten of the same overrides, ten different overrides, and forgetting a relationship.
• Figure 5: Simple dataset. Correlation of whether a block is fine-tuned with remaining accuracy, for editing a single tuple (left), editing ten tuples the same way (middle), and making ten different edits on ten different tuples (right).