Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Most Convolutional Networks Suffer from Small Adversarial Perturbations

Amit Daniely, Idan Mehalel

TL;DR

This work analyzes adversarial perturbations for convolutional networks with random initializations. It develops a Fourier/decomposition approach to bound the singular values of random convolutional layers, then links a large Jacobian to effective gradient-based attacks. The authors prove that, under mild smooth activation assumptions, a single gradient step suffices to produce adversarial examples at perturbation scales $\|f-f'\|=O(\|f\|/\sqrt{N_0})$, matching the known lower bounds up to constants. The results extend previous fully connected network analyses to CNNs, highlighting fundamental vulnerabilities of random CNNs and providing constructive insight via gradient dynamics and spectral bounds that could inform defenses and theoretical understanding.

Abstract

The existence of adversarial examples is relatively understood for random fully connected neural networks, but much less so for convolutional neural networks (CNNs). The recent work [Daniely, 2025] establishes that adversarial examples can be found in CNNs, in some non-optimal distance from the input. We extend over this work and prove that adversarial examples in random CNNs with input dimension $d$ can be found already in $\ell_2$-distance of order $\lVert x \rVert /\sqrt{d}$ from the input $x$, which is essentially the nearest possible. We also show that such adversarial small perturbations can be found using a single step of gradient descent. To derive our results we use Fourier decomposition to efficiently bound the singular values of a random linear convolutional operator, which is the main ingredient of a CNN layer. This bound might be of independent interest.

Most Convolutional Networks Suffer from Small Adversarial Perturbations

TL;DR

This work analyzes adversarial perturbations for convolutional networks with random initializations. It develops a Fourier/decomposition approach to bound the singular values of random convolutional layers, then links a large Jacobian to effective gradient-based attacks. The authors prove that, under mild smooth activation assumptions, a single gradient step suffices to produce adversarial examples at perturbation scales , matching the known lower bounds up to constants. The results extend previous fully connected network analyses to CNNs, highlighting fundamental vulnerabilities of random CNNs and providing constructive insight via gradient dynamics and spectral bounds that could inform defenses and theoretical understanding.

Abstract

The existence of adversarial examples is relatively understood for random fully connected neural networks, but much less so for convolutional neural networks (CNNs). The recent work [Daniely, 2025] establishes that adversarial examples can be found in CNNs, in some non-optimal distance from the input. We extend over this work and prove that adversarial examples in random CNNs with input dimension can be found already in -distance of order from the input , which is essentially the nearest possible. We also show that such adversarial small perturbations can be found using a single step of gradient descent. To derive our results we use Fourier decomposition to efficiently bound the singular values of a random linear convolutional operator, which is the main ingredient of a CNN layer. This bound might be of independent interest.
Paper Structure (24 sections, 26 theorems, 110 equations)

This paper contains 24 sections, 26 theorems, 110 equations.

Table of Contents

  1. Introduction
  2. Background and related work
  3. Our contribution
  4. Paper organization
  5. Preliminaries
  6. Architecture
  7. Initialization of the network
  8. Characters and Representations
  9. Results
  10. Proof sketch
  11. Proof sketch of Lemma \ref{['lem:intro-gradient']}
  12. Proof sketch of Lemma \ref{['lem:intro-gradient-robustness']}
  13. Proof sketch of Theorem \ref{['thm:sin-val-intro']}
  14. The singular values of a single linear convolutional layer
  15. Upper bound on singular values
  16. ...and 9 more sections

Key Result

Theorem 3.1

Fix $f \in L^2(G,\mathbb{R}^d)$ with $\lVert f \rVert_\infty \leq 1$. Then, with probability at least $0.95$, a single step of gradient descent of Euclidean length $O(1)$ starting from $f$ will reach $f' \in L^2(G,\mathbb{R}^d)$ such that $\mathsf{sign}(H_b(f')) \neq \mathsf{sign}(H_b(f))$.

Theorems & Definitions (46)

  • Theorem 3.1
  • Theorem 3.2
  • Lemma 4.1
  • Lemma 4.2
  • Lemma 4.3
  • Lemma 5.1
  • Theorem 5.2
  • Lemma 5.3
  • proof
  • Lemma 5.4
  • ...and 36 more