Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning the Mechanism of Catastrophic Forgetting: A Perspective from Gradient Similarity

Mutian Yang, Zisen Zhan, Yutong Chen, Haolin Li, Kaiwen Wang, Kaili Zheng, Yuguang Wang, Qi Wang, Jiandong Gao, Ji Wu

TL;DR

This work tackles catastrophic forgetting during knowledge injection in large language models by introducing a gradient-based framework that links forgetting to strongly negative gradient similarity. It reveals two neuron types—conflicting and collaborative—and proposes Collaborative Neural Learning (CNL) to freeze conflicting neurons while updating collaborative ones. Theoretical results show forgetting can be eliminated under an infinitesimal learning rate and known mastered set, and extensive experiments across multiple models, datasets, and optimizers validate zero forgetting in-set and large forgetting reductions out-of-set. The approach provides a principled, neuron-level mechanism for continual knowledge acquisition, with practical implications for robust knowledge editing and continual learning in LLMs.

Abstract

Catastrophic forgetting during knowledge injection severely undermines the continual learning capability of large language models (LLMs). Although existing methods attempt to mitigate this issue, they often lack a foundational theoretical explanation. We establish a gradient-based theoretical framework to explain catastrophic forgetting. We first prove that strongly negative gradient similarity is a fundamental cause of forgetting. We then use gradient similarity to identify two types of neurons: conflicting neurons that induce forgetting and account for 50%-75% of neurons, and collaborative neurons that mitigate forgetting and account for 25%-50%. Based on this analysis, we propose a knowledge injection method, Collaborative Neural Learning (CNL). By freezing conflicting neurons and updating only collaborative neurons, CNL theoretically eliminates catastrophic forgetting under an infinitesimal learning rate eta and an exactly known mastered set. Experiments on five LLMs, four datasets, and four optimizers show that CNL achieves zero forgetting in in-set settings and reduces forgetting by 59.1%-81.7% in out-of-set settings.

Learning the Mechanism of Catastrophic Forgetting: A Perspective from Gradient Similarity

TL;DR

This work tackles catastrophic forgetting during knowledge injection in large language models by introducing a gradient-based framework that links forgetting to strongly negative gradient similarity. It reveals two neuron types—conflicting and collaborative—and proposes Collaborative Neural Learning (CNL) to freeze conflicting neurons while updating collaborative ones. Theoretical results show forgetting can be eliminated under an infinitesimal learning rate and known mastered set, and extensive experiments across multiple models, datasets, and optimizers validate zero forgetting in-set and large forgetting reductions out-of-set. The approach provides a principled, neuron-level mechanism for continual knowledge acquisition, with practical implications for robust knowledge editing and continual learning in LLMs.

Abstract

Catastrophic forgetting during knowledge injection severely undermines the continual learning capability of large language models (LLMs). Although existing methods attempt to mitigate this issue, they often lack a foundational theoretical explanation. We establish a gradient-based theoretical framework to explain catastrophic forgetting. We first prove that strongly negative gradient similarity is a fundamental cause of forgetting. We then use gradient similarity to identify two types of neurons: conflicting neurons that induce forgetting and account for 50%-75% of neurons, and collaborative neurons that mitigate forgetting and account for 25%-50%. Based on this analysis, we propose a knowledge injection method, Collaborative Neural Learning (CNL). By freezing conflicting neurons and updating only collaborative neurons, CNL theoretically eliminates catastrophic forgetting under an infinitesimal learning rate eta and an exactly known mastered set. Experiments on five LLMs, four datasets, and four optimizers show that CNL achieves zero forgetting in in-set settings and reduces forgetting by 59.1%-81.7% in out-of-set settings.
Paper Structure (31 sections, 60 equations, 2 figures, 9 tables)

This paper contains 31 sections, 60 equations, 2 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Definition of Learning and Forgetting from the Perspective of Gradient
  3. Learning
  4. Forgetting
  5. Gradient Similarity Measures Catastrophic Forgetting
  6. Experiment Procedure
  7. Catastrophic Forgetting Happens during Knowledge Injection
  8. Strongly Negative Gradient Similarity Leads to Forgetting
  9. Conflicting Neurons Cause Catastrophic Forgetting
  10. Preventing Catastrophic Forgetting by Freezing Conflicting Neurons
  11. Establishment of CNL
  12. Extending CNL to More Complex Optimizers
  13. Extending CNL to More Complex Datasets
  14. Extending CNL to Out-of-Set Generalization
  15. Related Work
  16. ...and 16 more sections

Figures (2)

  • Figure 1: Conceptual illustration of our framework. We reveal the mechanism of catastrophic forgetting and propose a novel method to prevent its occurrence from a perspective of gradient similarity.
  • Figure 2: Learning and forgetting curves across four datasets and four optimizers. The rows represent SGD, Momentum, Adam, and AdamW. The inner labels indicate the metric type (Learning vs Forgetting) for the corresponding vertical position.