Robust Unlearnable Examples: Protecting Data Against Adversarial Learning

TL;DR

The paper tackles the problem of protecting data from being learned by adversarially trained models by showing that prior error-minimizing noise fails under adversarial training. It introduces robust error-minimizing noise (REM), learned via a min-min-max objective and stabilized with expectation over transformations (EOT). Empirical results across CIFAR-10/100 and ImageNet demonstrate REM’s effectiveness in preserving data unlearnability against adversarial learners, outperforming EM, TAP, and NTGA while highlighting the need for a defensive radius larger than the adversarial radius. This approach provides a practical pathway toward robust data protection in environments where training data may be illicitly used.

Abstract

The tremendous amount of accessible data in cyberspace face the risk of being unauthorized used for training deep learning models. To address this concern, methods are proposed to make data unlearnable for deep learning models by adding a type of error-minimizing noise. However, such conferred unlearnability is found fragile to adversarial training. In this paper, we design new methods to generate robust unlearnable examples that are protected from adversarial training. We first find that the vanilla error-minimizing noise, which suppresses the informative knowledge of data via minimizing the corresponding training loss, could not effectively minimize the adversarial training loss. This explains the vulnerability of error-minimizing noise in adversarial training. Based on the observation, robust error-minimizing noise is then introduced to reduce the adversarial training loss. Experiments show that the unlearnability brought by robust error-minimizing noise can effectively protect data from adversarial training in various scenarios. The code is available at \url{https://github.com/fshp971/robust-unlearnable-examples}.

Figures (8)

• Figure 1: We conduct adversarial training on data protected by different types of noise with varied adversarial perturbation radius $\rho_a$. EM denotes error-minimizing noise, TAP denotes targeted adversarial poisoning noise, NTGA denotes neural tangent generalization attack noise, and REM denotes robust error-minimizing noise. The curves of test accuracy vs. radius $\rho_a$ are plotted. The results show that as the radius $\rho_a$ increases, (1) the protection brought by EM, NTGA and TAP gradually become invalid, and (2) the proposed REM can still protect data against adversarial learners.
• Figure 2: The training loss curves of ERM training and adversarial training on CIFAR-10. Lower training losses suggest stronger unlearnability of data. The results show that: (1) error-minimizing noise could not reduce the training loss as effectively as that in ERM training; (2) robust error-minimizing noise can preserve the training loss in significantly low levels across various learning scenarios. These observations suggest that robust error-minimizing noise is more favorable in preventing data from being learned via adversarial training.
• Figure 3: Visualization results of different types of defensive noise as well as the correspondingly crafted examples. EM denotes error-minimizing noise, TAP denotes targeted adversarial poisoning noise, NTGA denotes neural tangent generalization attack noise, and REM denotes robust error-minimizing noise.
• Figure 4: Visualization results of CIFAR-10. Examples of data protected by error-minimizing noise (EM), targeted adversarial poisoning noise (TAP), neural tangent generalization attack noise (NTGA), and robust error-minimizing noise (REM). When $\rho_u$ is set as $8/255$ or $16/255$, the adversarial perturbation radius $\rho_a$ of REM is set as $4/255$ or $8/255$.
• Figure 5: Visualization results of CIFAR-100. Examples of data protected by error-minimizing noise (EM), targeted adversarial poisoning noise (TAP), neural tangent generalization attack noise (NTGA), and robust error-minimizing noise (REM). When $\rho_u$ is set as $8/255$ or $16/255$, the adversarial perturbation radius $\rho_a$ of REM is set as $4/255$ or $8/255$.