Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Get Confused Cautiously: Textual Sequence Memorization Erasure with Selective Entropy Maximization

Zhaohan Zhang, Ziquan Liu, Ioannis Patras

TL;DR

This work tackles the privacy and copyright concerns of Textual Sequence Memorization (TSM) in large language models by proposing EMSO, a reference-free framework that maximizes the entropy of the predictive distribution on a forget set to induce forgetting while preserving model utility. It achieves this by selectively updating a small set of weights determined by a contrastive gradient metric that balances forgetting direction with entropy optimization, avoiding reliance on memorized or retain data. Theoretical gradient analysis and extensive experiments on GPT-Neo variants show EMSO provides a superior erasure-utility trade-off compared with baselines, with greater stability (fewer collapses) and better preservation of language generation and reasoning across large forgetting requests. While EMSO advances scalable TSM erasure, it acknowledges limitations such as residual forgetfulness and memory requirements for the contrastive gradient computation, pointing to future work on more automatic privacy-data editing and efficiency improvements.

Abstract

Large Language Models (LLMs) have been found to memorize and recite some of the textual sequences from their training set verbatim, raising broad concerns about privacy and copyright issues when using LLMs. This Textual Sequence Memorization (TSM) phenomenon leads to a high demand to regulate LLM output to prevent it from generating certain memorized text to meet user requirements. However, our empirical study reveals that existing methods for TSM erasure fail to forget massive memorized samples without substantially jeopardizing the model utility. To achieve a better trade-off between the effectiveness of TSM erasure and model utility in LLMs, our paper proposes a new framework based on Entropy Maximization with Selective Optimization (EMSO), where the updated weights are chosen with a novel contrastive gradient metric without any participation of additional model or data. Our analysis shows that training with the entropy maximization loss has a more stable optimization process and better keeps model utility than existing methods. The contrastive gradient metric localizes the most influential weight for TSM erasure by taking both the gradient magnitude and direction into consideration. Extensive experiments across three model scales demonstrate that our method excels in handling large-scale forgetting requests while preserving model ability in language generation and reasoning.

Get Confused Cautiously: Textual Sequence Memorization Erasure with Selective Entropy Maximization

TL;DR

This work tackles the privacy and copyright concerns of Textual Sequence Memorization (TSM) in large language models by proposing EMSO, a reference-free framework that maximizes the entropy of the predictive distribution on a forget set to induce forgetting while preserving model utility. It achieves this by selectively updating a small set of weights determined by a contrastive gradient metric that balances forgetting direction with entropy optimization, avoiding reliance on memorized or retain data. Theoretical gradient analysis and extensive experiments on GPT-Neo variants show EMSO provides a superior erasure-utility trade-off compared with baselines, with greater stability (fewer collapses) and better preservation of language generation and reasoning across large forgetting requests. While EMSO advances scalable TSM erasure, it acknowledges limitations such as residual forgetfulness and memory requirements for the contrastive gradient computation, pointing to future work on more automatic privacy-data editing and efficiency improvements.

Abstract

Large Language Models (LLMs) have been found to memorize and recite some of the textual sequences from their training set verbatim, raising broad concerns about privacy and copyright issues when using LLMs. This Textual Sequence Memorization (TSM) phenomenon leads to a high demand to regulate LLM output to prevent it from generating certain memorized text to meet user requirements. However, our empirical study reveals that existing methods for TSM erasure fail to forget massive memorized samples without substantially jeopardizing the model utility. To achieve a better trade-off between the effectiveness of TSM erasure and model utility in LLMs, our paper proposes a new framework based on Entropy Maximization with Selective Optimization (EMSO), where the updated weights are chosen with a novel contrastive gradient metric without any participation of additional model or data. Our analysis shows that training with the entropy maximization loss has a more stable optimization process and better keeps model utility than existing methods. The contrastive gradient metric localizes the most influential weight for TSM erasure by taking both the gradient magnitude and direction into consideration. Extensive experiments across three model scales demonstrate that our method excels in handling large-scale forgetting requests while preserving model ability in language generation and reasoning.
Paper Structure (21 sections, 15 equations, 7 figures, 4 tables)

This paper contains 21 sections, 15 equations, 7 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Problem Definition
  5. Entropy Maximization
  6. Weight Selection with Contrastive Gradient
  7. Experiments
  8. Experiment Setup
  9. Test Model and Forget Set
  10. Comparison Methods
  11. Evaluation Metrics
  12. Experiment Results
  13. Ablation Study
  14. Discussion
  15. Conclusion
  16. ...and 6 more sections

Figures (7)

  • Figure 1: Illustration of erasure-utility trade-off with the example of three methods: Gradient Ascent (GA) jang2023knowledge, Deliberate Imagaination (DI) dong2024unmemorization and our EMSO. The top figure shows the exemplary erasure-utility trade-off with different cases. The bottom figure demonstrates the quantitative erasure-utility trade-off with the correlation between TSM metric (Memorization Accuracy, MA) and generation quality metric (MAUVE).
  • Figure 2: Framework of EMSO. We select the most significant weights for forgetting based on the gradient magnitude and direction so that the weight is effective at entropy maximization instead of data memorization.
  • Figure 3: Experiment results for language reasoning ability evaluation with GPT-Neo-125M (left), 1.3B (middle), and 2.7B (right) as target model. We report the average accuracy of the updated model on all six tasks in the legend. Our EMSO achieves the best performance on all three models compared with baselines.
  • Figure 4: Illustration of erasure-utility trade-off for different methods on GPT-Neo-125M. We vary the erasure strength for different methods. The one with high MAUVE and low Perplexity while low on MA and $\text{EL}_3$ is considered better (i.e., closer to the lower-right corner for MAUVE-MA and MAUVE-$\text{EL}_3$ figures and to the lower-left corner for Perplexity-MA and Perplexity-$\text{EL}_3$ figures). We do not plot the trade-off line for GD, KL, and GA here because they collapse before completing a single training epoch.
  • Figure 5: Illustration of the output change for memorized data and non-memorized data. We calculate the exact match between the original model and the updated model output given prefix from the forget set.
  • ...and 2 more figures