TruVRF: Towards Triple-Granularity Verification on Machine Unlearning

TL;DR

TruVRF addresses the lack of reliable verification for machine unlearning under data-forgetting regulations by introducing a non-invasive, model-sensitivity–based framework that operates at class-, volume-, and sample-level granularity. It defines three Unlearning-Metrics to detect Neglecting, Lazy, and Deceiving servers and validates them on CIFAR-10, Fashion-MNIST, and RAF-DB, with strong performance (over 90% accuracy for Metrics I and III) and modest volume inference error (4.8%–8.2% for Metric II). The approach is demonstrated to generalize to state-of-the-art unlearning frameworks like SISA (exact) and Amnesiac Unlearning (approximate), and is applicable to real-world face recognition scenarios. By leveraging gradient-based model sensitivity and shadow-model analyses, TruVRF offers a practical auditing tool that strengthens data owners’ rights and trust in MLaaS ecosystems.

Abstract

The concept of the right to be forgotten has led to growing interest in machine unlearning, but reliable validation methods are lacking, creating opportunities for dishonest model providers to mislead data contributors. Traditional invasive methods like backdoor injection are not feasible for legacy data. To address this, we introduce TruVRF, a non-invasive unlearning verification framework operating at class-, volume-, and sample-level granularities. TruVRF includes three Unlearning-Metrics designed to detect different types of dishonest servers: Neglecting, Lazy, and Deceiving. Unlearning-Metric-I checks class alignment, Unlearning-Metric-II verifies sample count, and Unlearning-Metric-III confirms specific sample deletion. Evaluations on three datasets show TruVRF's robust performance, with over 90% accuracy for Metrics I and III, and a 4.8% to 8.2% inference deviation for Metric II. TruVRF also demonstrates generalizability and practicality across various conditions and with state-of-the-art unlearning frameworks like SISA and Amnesiac Unlearning.

Figures (13)

• Figure 1: Machine unlearning workflow.
• Figure 2: A schematic view of threat model. ① A data contributor requests the data provider to delete the target data from all models that use the target data. ②. The data provider reviews the request of data contributor, and sends unlearning requests to corresponding model providers. ③. A malicious model provider executes dishonest unlearning in three ways. ④. The model provider returns the unlearned model to the data provider after unlearning for auditing. ⑤. The data provider responds with unlearning result to the data contributor.
• Figure 3: TruVRF overview.
• Figure 4: Process of Unlearned Class Verification.
• Figure 5: Process of Unlearned Volume Verification.