Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

"Forgetting" in Machine Learning and Beyond: A Survey

Alyssa Shuang Sha, Bernardo Pereira Nunes, Armin Haller

TL;DR

This survey investigates forgetting in ML by synthesising cross-disciplinary theories of forgetting (psychology, neuroscience, philosophy, ecology, linguistics) and mapping them onto ML taxonomies of content, recoverability, and extent. It advocates selective forgetting as a means to improve adaptability, generalisation, and privacy, and proposes a grounded-theory-based method to link theories across domains. Key contributions include a comprehensive forgetting taxonomy, a synthesis of active and passive forgetting approaches, and discussion of evaluation, ethics, and future directions. The work enables privacy-compliant, bias-aware, and efficient continual learning, offering a framework for responsible deployment of forgetting mechanisms in real-world systems.

Abstract

This survey investigates the multifaceted nature of forgetting in machine learning, drawing insights from neuroscientific research that posits forgetting as an adaptive function rather than a defect, enhancing the learning process and preventing overfitting. This survey focuses on the benefits of forgetting and its applications across various machine learning sub-fields that can help improve model performance and enhance data privacy. Moreover, the paper discusses current challenges, future directions, and ethical considerations regarding the integration of forgetting mechanisms into machine learning models.

"Forgetting" in Machine Learning and Beyond: A Survey

TL;DR

This survey investigates forgetting in ML by synthesising cross-disciplinary theories of forgetting (psychology, neuroscience, philosophy, ecology, linguistics) and mapping them onto ML taxonomies of content, recoverability, and extent. It advocates selective forgetting as a means to improve adaptability, generalisation, and privacy, and proposes a grounded-theory-based method to link theories across domains. Key contributions include a comprehensive forgetting taxonomy, a synthesis of active and passive forgetting approaches, and discussion of evaluation, ethics, and future directions. The work enables privacy-compliant, bias-aware, and efficient continual learning, offering a framework for responsible deployment of forgetting mechanisms in real-world systems.

Abstract

This survey investigates the multifaceted nature of forgetting in machine learning, drawing insights from neuroscientific research that posits forgetting as an adaptive function rather than a defect, enhancing the learning process and preventing overfitting. This survey focuses on the benefits of forgetting and its applications across various machine learning sub-fields that can help improve model performance and enhance data privacy. Moreover, the paper discusses current challenges, future directions, and ethical considerations regarding the integration of forgetting mechanisms into machine learning models.
Paper Structure (54 sections, 11 figures)

This paper contains 54 sections, 11 figures.

Table of Contents

  1. Introduction
  2. Background of Forgetting in different knowledge areas
  3. Forgetting in Psychology: ensure effective memorisation and emotional regulation
  4. Forgetting in Neuroscience: the magic of engram cells
  5. Forgetting in Education: a form to promote long-term learning
  6. Forgetting in Philosophy: multidimensional self-awareness and moral duty in humanhood
  7. Forgetting in Ecology: trade-offs in animal behaviour and plant habituation
  8. Forgetting in Linguistics: shaping language evolution and historical narratives
  9. The review of forgetting in machine learning
  10. Expand The linkage of forgetting across knowledge areas to machine learning
  11. Actual forgetting is rational forgetting
  12. Unlocking Potential: The Role of Forgetting in Adaptability, Generalisation, and Creativity
  13. How the process of forgetting is shaped?
  14. Challenges in forgetting
  15. A taxonomy of forgetting in machine learning
  16. ...and 39 more sections

Figures (11)

  • Figure 1: Scope and structure of the survey
  • Figure 2: The workflow of qualitative data analysis using grounded theory
  • Figure 3: Quantitative analysis on forgetting across different domains using grounded theory
  • Figure 4: A taxonomy of forgetting in machine learning
  • Figure 5: The workflow of active and passive forgetting approaches
  • ...and 6 more figures