Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Perturbation-Induced Linearization: Constructing Unlearnable Data with Solely Linear Classifiers

Jinlin Liu, Wei Chen, Xiaojin Zhang

TL;DR

Unlearnable examples aim to protect data by imperceptible perturbations, but existing approaches rely on costly deep surrogates. This paper introduces Perturbation-Induced Linearization (PIL), which uses a bias-free linear surrogate to craft perturbations under $\|\bm{\delta}_i\|_\infty \le \epsilon$ with $\epsilon = 8/255$ and a two-term loss to induce semantic obfuscation and a perturbation–label shortcut, producing an unlearnable dataset $\mathcal{D}_u = \{ (\bm{x}_i - \bm{\delta}_i^*, y_i) \}$. PIL achieves comparable or better protection than deep-surrogate methods while dramatically reducing compute time, and experiments show it generalizes across architectures/datasets and remains robust under augmentations and adversarial training. The paper also provides a theoretical partial-perturbation analysis and empirical evidence that unlearnable perturbations induce stronger linear behavior in DNNs, offering practical data protection and mechanistic insight.

Abstract

Collecting web data to train deep models has become increasingly common, raising concerns about unauthorized data usage. To mitigate this issue, unlearnable examples introduce imperceptible perturbations into data, preventing models from learning effectively. However, existing methods typically rely on deep neural networks as surrogate models for perturbation generation, resulting in significant computational costs. In this work, we propose Perturbation-Induced Linearization (PIL), a computationally efficient yet effective method that generates perturbations using only linear surrogate models. PIL achieves comparable or better performance than existing surrogate-based methods while reducing computational time dramatically. We further reveal a key mechanism underlying unlearnable examples: inducing linearization to deep models, which explains why PIL can achieve competitive results in a very short time. Beyond this, we provide an analysis about the property of unlearnable examples under percentage-based partial perturbation. Our work not only provides a practical approach for data protection but also offers insights into what makes unlearnable examples effective.

Perturbation-Induced Linearization: Constructing Unlearnable Data with Solely Linear Classifiers

TL;DR

Unlearnable examples aim to protect data by imperceptible perturbations, but existing approaches rely on costly deep surrogates. This paper introduces Perturbation-Induced Linearization (PIL), which uses a bias-free linear surrogate to craft perturbations under with and a two-term loss to induce semantic obfuscation and a perturbation–label shortcut, producing an unlearnable dataset . PIL achieves comparable or better protection than deep-surrogate methods while dramatically reducing compute time, and experiments show it generalizes across architectures/datasets and remains robust under augmentations and adversarial training. The paper also provides a theoretical partial-perturbation analysis and empirical evidence that unlearnable perturbations induce stronger linear behavior in DNNs, offering practical data protection and mechanistic insight.

Abstract

Collecting web data to train deep models has become increasingly common, raising concerns about unauthorized data usage. To mitigate this issue, unlearnable examples introduce imperceptible perturbations into data, preventing models from learning effectively. However, existing methods typically rely on deep neural networks as surrogate models for perturbation generation, resulting in significant computational costs. In this work, we propose Perturbation-Induced Linearization (PIL), a computationally efficient yet effective method that generates perturbations using only linear surrogate models. PIL achieves comparable or better performance than existing surrogate-based methods while reducing computational time dramatically. We further reveal a key mechanism underlying unlearnable examples: inducing linearization to deep models, which explains why PIL can achieve competitive results in a very short time. Beyond this, we provide an analysis about the property of unlearnable examples under percentage-based partial perturbation. Our work not only provides a practical approach for data protection but also offers insights into what makes unlearnable examples effective.
Paper Structure (36 sections, 1 theorem, 21 equations, 16 figures, 26 tables, 1 algorithm)

This paper contains 36 sections, 1 theorem, 21 equations, 16 figures, 26 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Proposed Method
  4. Problem Statement
  5. Perturbation-Induced Linearization
  6. Experiments
  7. Effectiveness of PIL Method on Different Datasets and Models
  8. PIL Method Against Common Countermeasures
  9. Data Augmentations.
  10. Adversarial Training.
  11. Time Comparison
  12. Understanding Perturbation-Driven Learning in DNNs.
  13. PIL do Improve the Linearity of DNNs
  14. Partial Perturbation: Why Doesn't Accuracy Drop Significantly?
  15. Conclusion
  16. ...and 21 more sections

Key Result

Theorem 1

Let $\theta$ be the model parameters, and let $\alpha$ be the fraction of perturbed training data. Denote $L_c(\theta)$ and $L_u(\theta)$ as the average loss on the clean data $\mathcal{D}_c$ and the perturbed data $\mathcal{D}_u$, respectively. For a learning rate $\eta$, the change in the clean da Under Assumption 1 (Gradient Orthogonality), i.e., $\nabla_\theta L_c(\theta_t) \cdot \nabla_\theta

Figures (16)

  • Figure 1: Illustration of the workflow of unlearnable examples. DNNs trained on the unlearnable data perform poorly on the clean test data. Results are reported for PIL on ImageNet-100.
  • Figure 2: Architecture illustration of PIL. We use $\oplus$ to denote inducing perturbations into images. Best viewed in color. Zoom in for details.
  • Figure 3: Clean test accuracy on CIFAR-10 under partial perturbation setting.
  • Figure 4: Accuracy curves with varying initial clean ratios ($\eta$). The star marker indicates the intersection point where the Clean curve crosses the PIL baseline. The x-axis (proportion) represents the total fraction of clean training data used relative to the entire dataset. (a)-(f) demonstrate the progressive impact of increasing $\eta$ from 0.1 to 0.8.
  • Figure 5: Effect of perturbations on prediction uncertainty and class confusion. Subfigures (a) and (c) show entropy of predictions on the clean test set, where higher values indicate greater uncertainty. Subfigures (b) and (d) present the corresponding confusion matrices.
  • ...and 11 more figures

Theorems & Definitions (1)

  • Theorem 1