Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Energy Propagation in Scattering Convolution Networks Can Be Arbitrarily Slow

Hartmut Führ, Max Getter

TL;DR

This work analyzes energy propagation in scattering convolution networks, showing that global exponential energy decay fails for wavelet-like filters in $L^2(\mathbb{R}^d)$ and that decay rates can be arbitrarily slow for generic signals. It offers a complementary positive theory: when signal spectra decay suitably and filters are frequency-localized (e.g., UFC or bandlimited wavelets), one can guarantee fast, in particular exponential, decay of the energy remainder $W_N$ across layers, quantified via a generalized Sobolev framework and explicit rates. Central tools include an approximate super-additivity lemma for frequency-separated signals, a constructive adversarial signal family to demonstrate slow propagation, and kernel-based integral bounds that connect Fourier content to energy decay. The results clarify the critical role of the interplay between signal Fourier decay and filter localization, providing practical guidance for when fast energy propagation can be expected (UFC/bandlimited cases) and highlighting the potential instability of decay in wavelet-based scattering for generic $L^2$ inputs.

Abstract

We analyze energy decay for deep convolutional neural networks employed as feature extractors, including Mallat's wavelet scattering transform. For time-frequency scattering transforms based on Gabor filters, previous work has established that energy decay is exponential for arbitrary square-integrable input signals. In contrast, our main results allow proving that this is false for wavelet scattering in arbitrary dimensions. Specifically, we show that the energy decay of wavelet and wavelet-like scattering transforms acting on generic square-integrable signals can be arbitrarily slow. Importantly, this slow decay behavior holds for dense subsets of $L^2(\mathbb{R}^d)$, indicating that rapid energy decay is generally an unstable property of signals. We complement these findings with positive results that allow us to infer fast (up to exponential) energy decay for generalized Sobolev spaces tailored to the frequency localization of the underlying filter bank. Both negative and positive results highlight that energy decay in scattering networks critically depends on the interplay between the respective frequency localizations of both the signal and the filters used.

Energy Propagation in Scattering Convolution Networks Can Be Arbitrarily Slow

TL;DR

This work analyzes energy propagation in scattering convolution networks, showing that global exponential energy decay fails for wavelet-like filters in and that decay rates can be arbitrarily slow for generic signals. It offers a complementary positive theory: when signal spectra decay suitably and filters are frequency-localized (e.g., UFC or bandlimited wavelets), one can guarantee fast, in particular exponential, decay of the energy remainder across layers, quantified via a generalized Sobolev framework and explicit rates. Central tools include an approximate super-additivity lemma for frequency-separated signals, a constructive adversarial signal family to demonstrate slow propagation, and kernel-based integral bounds that connect Fourier content to energy decay. The results clarify the critical role of the interplay between signal Fourier decay and filter localization, providing practical guidance for when fast energy propagation can be expected (UFC/bandlimited cases) and highlighting the potential instability of decay in wavelet-based scattering for generic inputs.

Abstract

We analyze energy decay for deep convolutional neural networks employed as feature extractors, including Mallat's wavelet scattering transform. For time-frequency scattering transforms based on Gabor filters, previous work has established that energy decay is exponential for arbitrary square-integrable input signals. In contrast, our main results allow proving that this is false for wavelet scattering in arbitrary dimensions. Specifically, we show that the energy decay of wavelet and wavelet-like scattering transforms acting on generic square-integrable signals can be arbitrarily slow. Importantly, this slow decay behavior holds for dense subsets of , indicating that rapid energy decay is generally an unstable property of signals. We complement these findings with positive results that allow us to infer fast (up to exponential) energy decay for generalized Sobolev spaces tailored to the frequency localization of the underlying filter bank. Both negative and positive results highlight that energy decay in scattering networks critically depends on the interplay between the respective frequency localizations of both the signal and the filters used.
Paper Structure (12 sections, 24 theorems, 209 equations, 6 figures, 1 table)

This paper contains 12 sections, 24 theorems, 209 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Motivation and related work
  3. Contributions
  4. Structure of the paper
  5. Notation
  6. A brief review of the windowed scattering transform.
  7. Slow Scattering Propagation
  8. Convergence Rates for Scattering Propagation
  9. Applications
  10. Scattering with UFC filters
  11. Scattering with Wavelets
  12. Concluding remarks

Key Result

Proposition 1.3

Every $f \in L^2({\mathbb{R}}^d)$ satisfies the energy decomposition, for all $N \in \mathbb{N}_0$, Consequently, which ensures the well-definedness of the operator $\mathcal{S}[\mathfrak{F}]: L^2({\mathbb{R}}^d) \to \ell^2(\mathcal{P}_\Psi;L^2({\mathbb{R}}^d))$. Moreover, holds if and only if

Figures (6)

  • Figure 1: Illustration of the operators $U[P],S[P;\chi]$ for the path set $P=\{e,\psi_1, \psi_1^\prime, (\psi_1,\psi_2),(\psi_1,\psi_2^\prime),(\psi_1^\prime,\psi_2)\}$ and output-generating filter $\chi$.
  • Figure 2: Indication of the architecture of a scattering network as described above, with filters $\psi_N,\psi_N^\prime \in \Psi$ corresponding to the $N$th network layer, $N \in \{1,2,3\}$. The function $\chi$ is the output-generating filter, which is (in our setup) the same across all layers. The residual energy $W_N(f)$ is the aggregated energy of the $N$th horizontal layer of the network, e.g., $W_2(f)=W_2[\Psi](f)=\left\lVert|f*\psi_1|*\psi_2\right\rVert_2^2+\Vert|f*\psi_1^\prime|*\psi_2^\prime\Vert_2^2+\cdots$.
  • Figure 3: Fourier transform of a sample function $f_0$, and of its dilations $f_2$, $f_4$, $f_6$, illustrating the frequency separation between the individual components of $f_E$ resulting from the dilation. Frequency separation is key to our construction of adversarial signals. In this illustration, $f_0=\frac{h}{\left\lVert h\right\rVert_2}$, and $\widehat{h}(\xi)=\mathds{1}_{(1,2)} \cdot \exp\left(-\frac{4}{1-(3-2\xi)^2}\right)$.
  • Figure 4: Approximation of the real part (left), and imaginary part (right) of $f_E$, where $E=(1/k)_{k\in \mathbb{N}}$, and $f_0=\mathcal{F}^{-1}(\mathds{1}_{[1,\frac{3}{2}]}-\mathds{1}_{[\frac{3}{2},2]})$. To avoid numerical issues arising from numbers being close to zero, and for the sake of illustration, we have explicitly chosen $m_k=2k$, $k \in \mathbb{N}$. The top figures indicate the global behavior of $f_E$, while the zoomed-in figures below suggest the wild local behavior of our semi-explicit constructions.
  • Figure 5: Illustration of the allowed regions for the frequency supports of two different filters from (possibly) two different families of high-pass filters. The parameter $\kappa$ (left $\kappa=2$, right $\kappa=3$) reflects the number of scales the filter may interfere with, and $\rho$ parameterizes the opening angle of the colored segment.
  • ...and 1 more figures

Theorems & Definitions (59)

  • Remark 1.1
  • Example 1.2
  • Proposition 1.3
  • Lemma 1.4
  • proof
  • Proposition 1.5
  • proof
  • Lemma 1.6
  • proof
  • Corollary 1.7
  • ...and 49 more