Robust Machine Unlearning for Quantized Neural Networks via Adaptive Gradient Reweighting with Similar Labels

TL;DR

This work tackles the problem of machine unlearning for quantized neural networks under privacy constraints. It introduces Q-MUL, a quantization-aware MU framework that (1) uses Similar Labels to reduce noise during data processing and (2) applies Adaptive Gradient Reweighting to balance updates from forgotten and retained data, with theoretical justification. Empirical results across CIFAR-10/100, SVHN, and Tiny-Imagenet with MobileNetV2 and ResNet-18 show Q-MUL achieving closer-to-retrain forgetting performance than prior methods, including under various QAT schemes. The approach offers a practical pathway for privacy-compliant model maintenance on resource-constrained devices, balancing unlearning effectiveness and computational efficiency.

Abstract

Model quantization enables efficient deployment of deep neural networks on edge devices through low-bit parameter representation, yet raises critical challenges for implementing machine unlearning (MU) under data privacy regulations. Existing MU methods designed for full-precision models fail to address two fundamental limitations in quantized networks: 1) Noise amplification from label mismatch during data processing, and 2) Gradient imbalance between forgotten and retained data during training. These issues are exacerbated by quantized models' constrained parameter space and discrete optimization. We propose Q-MUL, the first dedicated unlearning framework for quantized models. Our method introduces two key innovations: 1) Similar Labels assignment replaces random labels with semantically consistent alternatives to minimize noise injection, and 2) Adaptive Gradient Reweighting dynamically aligns parameter update contributions from forgotten and retained data. Through systematic analysis of quantized model vulnerabilities, we establish theoretical foundations for these mechanisms. Extensive evaluations on benchmark datasets demonstrate Q-MUL's superiority over existing approaches.

Figures (4)

• Figure 1: We conduct Random Labelsgolatkar2020eternal on the full-precision and quantized models. (a) The ratio of gradient norms between the forgotten data and the retained data in the loss. (b) Performance gaps of FA, RA, TA, and MIA with Retrain, AG is the average gap. The observations are from ResNet18 on CIFAR-10.
• Figure 2: The overview of Q-MUL. Before performing the unlearning process on the model, Q-MUL first selects the label that is closest to the true label in the current output of the model as the new label for the data samples that need to be forgotten. During the unlearning process, we adaptively calculate the weights of the forgotten and retained data based on their gradient norms.
• Figure 3: Performance gaps of Full-precision ResNet18 on CIFAR-100. The unlearning scenario is random data forgetting (10%). A shorter bar (smaller gap) indicates performance is closer to that of the Retrained model. The average gap for each method is calculated by dividing the values in the Summary bar chart by 4.
• Figure 4: Efficiency Analysis. The experimental scenario involves MobileNetV2 performing random data forgetting (10%) on CIFAR-100.