Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Comparative Study of Machine Unlearning Techniques for Image and Text Classification Models

Omar M. Safa, Mahmoud M. Abdelaziz, Mustafa Eltawy, Mohamed Mamdouh, Moamen Gharib, Salaheldin Eltenihy, Nagia M. Ghanem, Mohamed M. Ismail

TL;DR

The paper addresses the need to remove individuals' data from trained models to satisfy GDPR/CCPA while preserving utility. It proposes a unified benchmark that evaluates six unlearning techniques (SSD, Incompetent Teacher, SCRUB, UNSIR, Mislabel, and retraining as a baseline) across image (ResNet18, ViT) and text (MARBERT) tasks using CIFAR-10/100 and HARD. The findings show that Incompetent Teacher delivers the most balanced forgetting performance across full-class and random forgetting, while Mislabel Unlearning excels for full-class forgetting under transfer learning; SCRUB performs best on CNNs but struggles with text, and SSD offers fast, generally strong forgetting with high retainment in some cases. The results provide practical guidance for deploying unlearning under privacy regulations and identify avenues for improving cross-domain unlearning capabilities.

Abstract

Machine Unlearning has emerged as a critical area in artificial intelligence, addressing the need to selectively remove learned data from machine learning models in response to data privacy regulations. This paper provides a comprehensive comparative analysis of six state-of-theart unlearning techniques applied to image and text classification tasks. We evaluate their performance, efficiency, and compliance with regulatory requirements, highlighting their strengths and limitations in practical scenarios. By systematically analyzing these methods, we aim to provide insights into their applicability, challenges,and tradeoffs, fostering advancements in the field of ethical and adaptable machine learning.

A Comparative Study of Machine Unlearning Techniques for Image and Text Classification Models

TL;DR

The paper addresses the need to remove individuals' data from trained models to satisfy GDPR/CCPA while preserving utility. It proposes a unified benchmark that evaluates six unlearning techniques (SSD, Incompetent Teacher, SCRUB, UNSIR, Mislabel, and retraining as a baseline) across image (ResNet18, ViT) and text (MARBERT) tasks using CIFAR-10/100 and HARD. The findings show that Incompetent Teacher delivers the most balanced forgetting performance across full-class and random forgetting, while Mislabel Unlearning excels for full-class forgetting under transfer learning; SCRUB performs best on CNNs but struggles with text, and SSD offers fast, generally strong forgetting with high retainment in some cases. The results provide practical guidance for deploying unlearning under privacy regulations and identify avenues for improving cross-domain unlearning capabilities.

Abstract

Machine Unlearning has emerged as a critical area in artificial intelligence, addressing the need to selectively remove learned data from machine learning models in response to data privacy regulations. This paper provides a comprehensive comparative analysis of six state-of-theart unlearning techniques applied to image and text classification tasks. We evaluate their performance, efficiency, and compliance with regulatory requirements, highlighting their strengths and limitations in practical scenarios. By systematically analyzing these methods, we aim to provide insights into their applicability, challenges,and tradeoffs, fostering advancements in the field of ethical and adaptable machine learning.
Paper Structure (21 sections, 4 equations, 17 tables)

This paper contains 21 sections, 4 equations, 17 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Models and Datasets
  5. Evaluation Metrics
  6. Relative Retain Accuracy ($A_r$)
  7. Relative Forget Accuracy ($A_f$)
  8. Membership Inference Attack (MIA)
  9. Zero Retrain Forgetting (ZRF)
  10. Computation Time
  11. Experiments
  12. Hyperparameters
  13. Results
  14. Full Class Forgetting
  15. ResNet18 Results
  16. ...and 6 more sections