Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Theory of periodic convolutional neural network

Yuqing Liu

TL;DR

This paper analyzes a variant of convolutional neural networks with periodic boundary conditions on the torus $\mathbb{T}^d$ and establishes a sharp expressivity boundary for ridge-function approximation. Using a combination of constructive deep-network realizations and Fourier/lattice arguments, it shows that periodic-CNNs can approximate ridge functions depending on $d-1$ linear directions, but provably cannot approximate ridges depending on $d-2$ or fewer directions. The results provide a precise separation in the expressivity of CNN variants and suggest practical relevance for high-dimensional, structured data on wrapped domains, such as spectral analysis, physics-informed learning, and materials science. Overall, the work advances the theoretical understanding of CNN approximation and identifies a practically meaningful class of architectures with near-full but not universal ridge-function expressivity.

Abstract

We introduce a novel convolutional neural network architecture, termed the \emph{periodic CNN}, which incorporates periodic boundary conditions into the convolutional layers. Our main theoretical contribution is a rigorous approximation theorem: periodic CNNs can approximate ridge functions depending on $d-1$ linear variables in a $d$-dimensional input space, while such approximation is impossible in lower-dimensional ridge settings ($d-2$ or fewer variables). This result establishes a sharp characterization of the expressive power of periodic CNNs. Beyond the theory, our findings suggest that periodic CNNs are particularly well-suited for problems where data naturally admits a ridge-like structure of high intrinsic dimension, such as image analysis on wrapped domains, physics-informed learning, and materials science. The work thus both expands the mathematical foundation of CNN approximation theory and highlights a class of architectures with surprising and practically relevant approximation capabilities.

Theory of periodic convolutional neural network

TL;DR

This paper analyzes a variant of convolutional neural networks with periodic boundary conditions on the torus and establishes a sharp expressivity boundary for ridge-function approximation. Using a combination of constructive deep-network realizations and Fourier/lattice arguments, it shows that periodic-CNNs can approximate ridge functions depending on linear directions, but provably cannot approximate ridges depending on or fewer directions. The results provide a precise separation in the expressivity of CNN variants and suggest practical relevance for high-dimensional, structured data on wrapped domains, such as spectral analysis, physics-informed learning, and materials science. Overall, the work advances the theoretical understanding of CNN approximation and identifies a practically meaningful class of architectures with near-full but not universal ridge-function expressivity.

Abstract

We introduce a novel convolutional neural network architecture, termed the \emph{periodic CNN}, which incorporates periodic boundary conditions into the convolutional layers. Our main theoretical contribution is a rigorous approximation theorem: periodic CNNs can approximate ridge functions depending on linear variables in a -dimensional input space, while such approximation is impossible in lower-dimensional ridge settings ( or fewer variables). This result establishes a sharp characterization of the expressive power of periodic CNNs. Beyond the theory, our findings suggest that periodic CNNs are particularly well-suited for problems where data naturally admits a ridge-like structure of high intrinsic dimension, such as image analysis on wrapped domains, physics-informed learning, and materials science. The work thus both expands the mathematical foundation of CNN approximation theory and highlights a class of architectures with surprising and practically relevant approximation capabilities.

Paper Structure

This paper contains 10 sections, 9 theorems, 71 equations.

Table of Contents

  1. Introduction
  2. Periodic CNN Architecture
  3. Prelims
  4. Periodic CNN
  5. Main Result
  6. Approximation of $(d-1)$-dimensional ridge functions
  7. Impossibility for $(d-2)$-dimensional ridge functions
  8. A concrete counterexample
  9. Summary.
  10. Further Questions and Future Work

Key Result

Theorem 3.1.1

Given weight vectors $w_{1}, w_{2}, \cdots w_{J}$ and bias vectors $b_{1}, b_{2}, \cdots b_{J}$, and let $W \cdot x = y$, where $W \in \mathbb{R}^{d}$, $x \in \mathbb{R}^{d}$. We define the relation $\preceq$ such that if and only if $\Longleftrightarrow$

Theorems & Definitions (16)

  • Definition 2.2.1
  • Definition 2.2.2
  • Theorem 3.1.1
  • Lemma 3.1.2
  • Theorem 3.1.3
  • proof
  • Definition 3.2.1: Network lattice and associated subspace
  • Lemma 3.2.2: Fourier representation of a ridge function
  • Lemma 3.2.3: One-dimensional lattice inclusion
  • proof
  • ...and 6 more