Beyond Uniform Deletion: A Data Value-Weighted Framework for Certified Machine Unlearning
Lisong He, Yi Yang, Xiangyu Chang
TL;DR
The paper tackles privacy-preserving data deletion by addressing heterogeneity in data value, which prior unlearning methods often overlook. It introduces Data Value-Weighted Unlearning (DVWU), a framework that computes data values and maps them to weights, then integrates these weights into unlearning updates—grounded in the one-step Newton update—with two certified unlearning variants based on output and objective perturbation. DVWU demonstrates improved predictive performance and robustness over standard unlearning and retraining across linear and deep models, with theoretical bounds on gradient residuals and parameter gaps that support certification. The framework is flexible and extensible to gradient-based deep unlearning, offering practical, efficient privacy-preserving updates suitable for real-world deployment. The work highlights the importance of data-value aware strategies in maintaining utility while satisfying deletion requests, and it outlines future directions for scaling DVWU to large-scale models such as LLMs.
Abstract
As the right to be forgotten becomes legislated worldwide, machine unlearning mechanisms have emerged to efficiently update models for data deletion and enhance user privacy protection. However, existing machine unlearning algorithms frequently neglect the fact that different data points may contribute unequally to model performance (i.e., heterogeneous data values). Treat them equally in machine unlearning procedure can potentially degrading the performance of updated models. To address this limitation, we propose Data Value-Weighted Unlearning (DVWU), a general unlearning framework that accounts for data value heterogeneity into the unlearning process. Specifically, we design a weighting strategy based on data values, which are then integrated into the unlearning procedure to enable differentiated unlearning for data points with varying utility to the model. The DVWU framework can be broadly adapted to various existing machine unlearning methods. We use the one-step Newton update as an example for implementation, developing both output and objective perturbation algorithms to achieve certified unlearning. Experiments on both synthetic and real-world datasets demonstrate that our methods achieve superior predictive performance and robustness compared to conventional unlearning approaches. We further show the extensibility of our framework on gradient ascent method by incorporating the proposed weighting strategy into the gradient terms, highlighting the adaptability of DVWU for broader gradient-based deep unlearning methods.