Unlearning in- vs. out-of-distribution data in LLMs under gradient-based method

TL;DR

This work formalizes a metric to evaluate unlearning quality in generative models, and uses it to assess the trade-offs between unlearning quality and performance and demonstrates that unlearning out-of-distribution examples requires more unlearning steps but overall presents a better trade-off overall.

Abstract

Machine unlearning aims to solve the problem of removing the influence of selected training examples from a learned model. Despite the increasing attention to this problem, it remains an open research question how to evaluate unlearning in large language models (LLMs), and what are the critical properties of the data to be unlearned that affect the quality and efficiency of unlearning. This work formalizes a metric to evaluate unlearning quality in generative models, and uses it to assess the trade-offs between unlearning quality and performance. We demonstrate that unlearning out-of-distribution examples requires more unlearning steps but overall presents a better trade-off overall. For in-distribution examples, however, we observe a rapid decay in performance as unlearning progresses. We further evaluate how example's memorization and difficulty affect unlearning under a classical gradient ascent-based approach.

Figures (23)

• Figure 1: In- vs. out-of-distribution (canary) trade-off. Trade-off between the generalized exposure (Exposure) and the task performance (BLEU score) when unlearning the subject model ($\theta_S$) at 45,000 steps, with in-distribution sets and canary sets repeated 100 times (left), 10 times (middle), and 1 time (right) during training.
• Figure 2: Distributions of perplexities. Perplexities of different sets of in-distribution examples under the subject model (before unlearning, post-unlearning and when exposure is low) and the reference model. Columns left to right: in-distribution example perplexities when the subject model was trained by repeating these examples 100 times (left), 10 times (middle), 1 time (right).
• Figure 3: Per-sample memorization vs. difficulty. The memorization vs. difficulty for each sample in the forgets sets that repeat $\times 1$, and $\times 100$. Difficulty and memorization become correlated with number of repeats.
• Figure 4: Difficulty vs. trade-offs. Measure the trade-offs of unlearning examples of low, medium and high difficulty. Harder examples have slightly better trade-offs.
• Figure 5: Unlearning affects the average exposure of similar examples.

Theorems & Definitions (1)

• Definition 2.1