Machine Unlearning for Document Classification

TL;DR

This work investigates machine unlearning for document classification under privacy constraints, proposing a practical setting where only a small fraction ($10\%$) of training data is retained on a remote server. It introduces a label-guided sample generator to synthesize forget data, enabling immediate forgetting without requiring access to real forget samples, and compares several unlearning baselines (Retrain, Fine-tune, Random Label) across subset selection strategies. Experiments on RVL-CDIP show that generated forget data can approximate real forget data in effectiveness, achieving strong retention while reducing forget information, particularly under restricted data and iteration budgets. Overall, the work demonstrates a viable, privacy-preserving approach to document analysis with tangible implications for real-world AI services.

Abstract

Document understanding models have recently demonstrated remarkable performance by leveraging extensive collections of user documents. However, since documents often contain large amounts of personal data, their usage can pose a threat to user privacy and weaken the bonds of trust between humans and AI services. In response to these concerns, legislation advocating ``the right to be forgotten" has recently been proposed, allowing users to request the removal of private information from computer systems and neural network models. A novel approach, known as machine unlearning, has emerged to make AI models forget about a particular class of data. In our research, we explore machine unlearning for document classification problems, representing, to the best of our knowledge, the first investigation into this area. Specifically, we consider a realistic scenario where a remote server houses a well-trained model and possesses only a small portion of training data. This setup is designed for efficient forgetting manipulation. This work represents a pioneering step towards the development of machine unlearning methods aimed at addressing privacy concerns in document analysis applications. Our code is publicly available at \url{https://github.com/leitro/MachineUnlearning-DocClassification}.

Figures (9)

• Figure 1: Architecture of the proposed backbone document classifier. Our model processes 224 $\times$ 224 images to classify them, the output is a probability distribution over the 16 document classes of the RVL-CDIP dataset.
• Figure 2: Architecture of label-guided sample generation method. We train the projector model to project the one-hot vector of class labels $y$ and random noise vectors $z$ into synthetic samples that can then be used to replace the need of real data for machine unlearning.
• Figure 3: Original.
• Figure 4: Retain (RT).
• Figure 5: Fine-tune (FT).