Parameter-tuning-free data entry error unlearning with adaptive selective synaptic dampening

TL;DR

The paper tackles the problem of data-entry label errors degrading model performance and the high cost of full retraining. It introduces Adaptive Selective Synaptic Dampening (ASSD), a parameter-free extension of Selective Synaptic Dampening (SSD) that automatically selects a forgetting aggressiveness using a percentile of relative Fisher Information importances, enabling forgetting of the erroneous data without tuning. The authors validate ASSD on standard unlearning benchmarks with ResNet18 and Vision Transformer, and demonstrate its practical viability on a real-world supply-chain delay dataset with varying label-error rates, where it matches or outperforms fine-tuning and approaches retraining at low error rates. This work enables robust, scalable data-entry error unlearning in industrial settings, reducing computational costs while preserving performance on correctly labeled data ($D_r$).

Abstract

Data entry constitutes a fundamental component of the machine learning pipeline, yet it frequently results in the introduction of labelling errors. When a model has been trained on a dataset containing such errors its performance is reduced. This leads to the challenge of efficiently unlearning the influence of the erroneous data to improve the model performance without needing to completely retrain the model. While model editing methods exist for cases in which the correct label for a wrong entry is known, we focus on the case of data entry errors where we do not know the correct labels for the erroneous data. Our contribution is twofold. First, we introduce an extension to the selective synaptic dampening unlearning method that removes the need for parameter tuning, making unlearning accessible to practitioners. We demonstrate the performance of this extension, adaptive selective synaptic dampening (ASSD), on various ResNet18 and Vision Transformer unlearning tasks. Second, we demonstrate the performance of ASSD in a supply chain delay prediction problem with labelling errors using real-world data where we randomly introduce various levels of labelling errors. The application of this approach is particularly compelling in industrial settings, such as supply chain management, where a significant portion of data entry occurs manually through Excel sheets, rendering it error-prone. ASSD shows strong performance on general unlearning benchmarks and on the error correction problem where it outperforms fine-tuning for error correction.

Figures (5)

• Figure 1: Unlearning data entry errors from a model trained on data including label errors can achieve model behaviour similar to that of a model trained solely on the correct data.
• Figure 2: SSD hyperparameter search on the CIFAR100 rocket fullclass unlearning task on ResNet18 using Optuna highlights the high importance of $\alpha$ whereas $\lambda$ values show little impact on the unlearning performance as quantified by the MIA and retain accuracy values.
• Figure 3: A simplified visualization of the unlearning plateau where model accuracy and MIA both are at near ideal values. Setting $\alpha$ above or below the plateau leads to significant drops in the desired performance of MIA or accuracy.
• Figure 4: Model accuracy improvement due to unlearning compared with the baseline model trained on data with random errors for 100 random scenarios using a neural network with three layers of 100 neurons each.
• Figure 5: Model accuracy improvement due to unlearning compared with the baseline model trained on data with 1% random errors and a retrained model on data excluding the errors. 100 error scenarios with randomly chosen observations to be mislabeled are shown ordered by the accuracy of the baseline model on the test data.