Machine Unlearning: Solutions and Challenges

TL;DR

Machine unlearning addresses privacy, security, and adaptability by enabling selective removal of training data influence from trained models. It categorizes methods into exact unlearning (full data removal) and approximate unlearning (efficient but partial removal), and provides a taxonomy, critical analysis, and open problems. The survey covers SISA-based exact unlearning, graph and FL adaptations, influence-function and re-optimization based approximations, and novel techniques for non-convex models and dynamic data, highlighting storage, assumptions, and verification as key challenges. It delineates future directions toward verifiable, scalable, and privacy-preserving unlearning, with potential impact on trustworthy and adaptive AI systems.

Abstract

Machine learning models may inadvertently memorize sensitive, unauthorized, or malicious data, posing risks of privacy breaches, security vulnerabilities, and performance degradation. To address these issues, machine unlearning has emerged as a critical technique to selectively remove specific training data points' influence on trained models. This paper provides a comprehensive taxonomy and analysis of the solutions in machine unlearning. We categorize existing solutions into exact unlearning approaches that remove data influence thoroughly and approximate unlearning approaches that efficiently minimize data influence. By comprehensively reviewing solutions, we identify and discuss their strengths and limitations. Furthermore, we propose future directions to advance machine unlearning and establish it as an essential capability for trustworthy and adaptive machine learning models. This paper provides researchers with a roadmap of open problems, encouraging impactful contributions to address real-world needs for selective data removal.

Figures (2)

• Figure 1: Illustration of Naive Retraining and Machine Unlearning
• Figure 2: SISA Framework

Theorems & Definitions (1)

• Definition 1: Naive Retraining