Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models

George-Octavian Barbulescu, Peter Triantafillou

TL;DR

This paper tackles memorized-data unlearning in large language models by arguing that forgetting should be per-example rather than aggregate. It defines an exact-memorization metric g(x) and an outlier-based Forgetting metric c(D_f), then proposes two main unlearning methods—Selective Gradient Ascent (SGA) and Task Arithmetic for Unlearning (TAU)—with TA as a related detoxification technique. A novel MIAU privacy attack demonstrates weaknesses in prior aggregate-focused approaches and motivates the new per-example strategy. Empirical evaluation on GPT-Neo variants shows that SGA and TAU achieve superior privacy-utility trade-offs across model sizes and forget-set scales, with TA excelling in small models and SGATAU proving robust for larger models. The work advances practical, fine-grained unlearning and highlights the importance of monitoring per-example memorization to mitigate privacy and copyright risks in LLMs.

Abstract

LLMs have been found to memorize training textual sequences and regurgitate verbatim said sequences during text generation time. This fact is known to be the cause of privacy and related (e.g., copyright) problems. Unlearning in LLMs then takes the form of devising new algorithms that will properly deal with these side-effects of memorized data, while not hurting the model's utility. We offer a fresh perspective towards this goal, namely, that each textual sequence to be forgotten should be treated differently when being unlearned based on its degree of memorization within the LLM. We contribute a new metric for measuring unlearning quality, an adversarial attack showing that SOTA algorithms lacking this perspective fail for privacy, and two new unlearning methods based on Gradient Ascent and Task Arithmetic, respectively. A comprehensive performance evaluation across an extensive suite of NLP tasks then mapped the solution space, identifying the best solutions under different scales in model capacities and forget set sizes and quantified the gains of the new approaches.

To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models

TL;DR

This paper tackles memorized-data unlearning in large language models by arguing that forgetting should be per-example rather than aggregate. It defines an exact-memorization metric g(x) and an outlier-based Forgetting metric c(D_f), then proposes two main unlearning methods—Selective Gradient Ascent (SGA) and Task Arithmetic for Unlearning (TAU)—with TA as a related detoxification technique. A novel MIAU privacy attack demonstrates weaknesses in prior aggregate-focused approaches and motivates the new per-example strategy. Empirical evaluation on GPT-Neo variants shows that SGA and TAU achieve superior privacy-utility trade-offs across model sizes and forget-set scales, with TA excelling in small models and SGATAU proving robust for larger models. The work advances practical, fine-grained unlearning and highlights the importance of monitoring per-example memorization to mitigate privacy and copyright risks in LLMs.

Abstract

LLMs have been found to memorize training textual sequences and regurgitate verbatim said sequences during text generation time. This fact is known to be the cause of privacy and related (e.g., copyright) problems. Unlearning in LLMs then takes the form of devising new algorithms that will properly deal with these side-effects of memorized data, while not hurting the model's utility. We offer a fresh perspective towards this goal, namely, that each textual sequence to be forgotten should be treated differently when being unlearned based on its degree of memorization within the LLM. We contribute a new metric for measuring unlearning quality, an adversarial attack showing that SOTA algorithms lacking this perspective fail for privacy, and two new unlearning methods based on Gradient Ascent and Task Arithmetic, respectively. A comprehensive performance evaluation across an extensive suite of NLP tasks then mapped the solution space, identifying the best solutions under different scales in model capacities and forget set sizes and quantified the gains of the new approaches.
Paper Structure (32 sections, 7 equations, 12 figures, 9 tables)

This paper contains 32 sections, 7 equations, 12 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Memorization in Large Language Models
  4. Machine Unlearning for Memorized Data
  5. Related Work
  6. The New Approach
  7. Quantifying Memorization in LLMs
  8. Selective Unlearning
  9. Selective Gradient Ascent
  10. Task Arithmetic for Unlearning
  11. Experimental Evaluation
  12. General setup
  13. Model Family & Forget Set
  14. Baselines
  15. Evaluation Metrics
  16. ...and 17 more sections

Figures (12)

  • Figure 1: The distribution of exact memorization scores across the forget set $D_f$ when the average memorization threshold is $\leq 33\%$ (as exhibited by GPT-Neo with 1.3B parameters over unseen/arbitrary textual sequences jang2022knowledge). While \ref{['ineq']} holds, 5 (8) samples from $D_f$ have an exact memorization score of over $60\%$ (under $10\%)$.
  • Figure 2: Average F1 scores for dialogue tasks. Unlearning 8 (left), 32 (middle), and 128 (right) samples. On the 125M model. The key discovery here is that TA and/or TAU perform best in small LLMs.
  • Figure 3: Average F1 scores for dialogue tasks. Unlearning 32 samples. On the 1.3B (left) and 2.7B (right) models. The key discovery here is that SGA catches up and surpasses the rest for larger models.
  • Figure 4: From left to right: the loss distribution across 256 textual sequences by querying the original model (1.3B GPT-Neo checkpoint) and the model after unlearning. On the x-axis we see the distance of training samples (members) to their artificial neighbours (\ref{['miau']}). We use 128 samples from the enron email dataset shetty2004enron as members and 128 samples from the SAMSum training dataset gliwa-etal-2019-samsum as nonmembers. After unlearning the members with gradient ascent, they become easily extractable (less overlap with nonmembers) as the distance between their loss and that of their neighbours' becomes increasingly large.
  • Figure 5: From left to right: the ROC curves for the experiment setup in \ref{['MIA']} for the original model and the unlearned model. Remarkably, there is a $50\%$ increase in the success of the adversarial attack after unlearning, from a true positive rate (TPR) of $60\%$ to a TPR of $90\%$, at a false positive rate (FPR) of $0\%$. And there is a clear difference on FPR in the FNR region [0, 0.2], which includes the regions we should attend to, according to Carlini et al carlini2022membership.
  • ...and 7 more figures

Theorems & Definitions (6)

  • Definition 4.1
  • Definition 4.2
  • Definition 4.3
  • Definition 4.4
  • Definition 7.1
  • Definition 7.2