Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MPRU: Modular Projection-Redistribution Unlearning as Output Filter for Classification Pipelines

Minyi Peng, Darian Gunamardi, Ivan Tjuawinata, Kwok-Yan Lam

TL;DR

MPRU tackles scalable machine unlearning by reframing classifier training as an inductive, sequential process and introducing a modular output filter that reverses the last learning step. The core idea is to project the pretrained model's confidence vector onto a hyperplane orthogonal to the forget-set signal, then redistribute the discarded mass across retained classes to form a new, $n-1$ output classifier without modifying model parameters. The authors provide a detailed algorithmic formulation, asymptotic complexity analysis, and extensive experiments on CIFAR-10/100 and Covertype, showing close agreement with fully retrained baselines in both accuracy and output distributions while achieving significant runtime savings. This work demonstrates the practicality of model-agnostic, post-processing unlearning that can be integrated into existing pipelines with minimal changes, enabling scalable compliance in real-world systems.

Abstract

As a new and promising approach, existing machine unlearning (MU) works typically emphasize theoretical formulations or optimization objectives to achieve knowledge removal. However, when deployed in real-world scenarios, such solutions typically face scalability issues and have to address practical requirements such as full access to original datasets and model. In contrast to the existing approaches, we regard classification training as a sequential process where classes are learned sequentially, which we call \emph{inductive approach}. Unlearning can then be done by reversing the last training sequence. This is implemented by appending a projection-redistribution layer in the end of the model. Such an approach does not require full access to the original dataset or the model, addressing the challenges of existing methods. This enables modular and model-agnostic deployment as an output filter into existing classification pipelines with minimal alterations. We conducted multiple experiments across multiple datasets including image (CIFAR-10/100 using CNN-based model) and tabular datasets (Covertype using tree-based model). Experiment results show consistently similar output to a fully retrained model with a high computational cost reduction. This demonstrates the applicability, scalability, and system compatibility of our solution while maintaining the performance of the output in a more practical setting.

MPRU: Modular Projection-Redistribution Unlearning as Output Filter for Classification Pipelines

TL;DR

MPRU tackles scalable machine unlearning by reframing classifier training as an inductive, sequential process and introducing a modular output filter that reverses the last learning step. The core idea is to project the pretrained model's confidence vector onto a hyperplane orthogonal to the forget-set signal, then redistribute the discarded mass across retained classes to form a new, output classifier without modifying model parameters. The authors provide a detailed algorithmic formulation, asymptotic complexity analysis, and extensive experiments on CIFAR-10/100 and Covertype, showing close agreement with fully retrained baselines in both accuracy and output distributions while achieving significant runtime savings. This work demonstrates the practicality of model-agnostic, post-processing unlearning that can be integrated into existing pipelines with minimal changes, enabling scalable compliance in real-world systems.

Abstract

As a new and promising approach, existing machine unlearning (MU) works typically emphasize theoretical formulations or optimization objectives to achieve knowledge removal. However, when deployed in real-world scenarios, such solutions typically face scalability issues and have to address practical requirements such as full access to original datasets and model. In contrast to the existing approaches, we regard classification training as a sequential process where classes are learned sequentially, which we call \emph{inductive approach}. Unlearning can then be done by reversing the last training sequence. This is implemented by appending a projection-redistribution layer in the end of the model. Such an approach does not require full access to the original dataset or the model, addressing the challenges of existing methods. This enables modular and model-agnostic deployment as an output filter into existing classification pipelines with minimal alterations. We conducted multiple experiments across multiple datasets including image (CIFAR-10/100 using CNN-based model) and tabular datasets (Covertype using tree-based model). Experiment results show consistently similar output to a fully retrained model with a high computational cost reduction. This demonstrates the applicability, scalability, and system compatibility of our solution while maintaining the performance of the output in a more practical setting.

Paper Structure

This paper contains 21 sections, 13 equations, 6 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Notations
  4. General Class Unlearning Scheme and Evaluation Metrics
  5. Problem Statement
  6. Assumptions
  7. Discussion on Inductivity in Learning
  8. Modular Projection-Redistribution Unlearning
  9. Unlearning Scheme
  10. Projection
  11. Redistribution
  12. Distributing $\mathbf{c}_{x,u}$
  13. Adjusting $\mathbf{c}_{x,r}$
  14. Entire Unlearning Formula
  15. Asymptotic Complexity Analysis
  16. ...and 6 more sections

Figures (6)

  • Figure 1: Inductivity in Learning
  • Figure 2: Overview of the proposed Modular Projection-Redistribution Unlearning (MPRU) framework
  • Figure 3: Distribute Factor $\tilde{\mathbf{c}_{u,r}}^P$
  • Figure 4: Prediction Comparison for CIFAR-10 between Full Retraining and MPRU with removed class 3 "cat".
  • Figure 5: Prediction Comparison for CIFAR-100 between Full Retraining and MPRU with removed class 35 "girl". Select few classes are shown for brevity.
  • ...and 1 more figures