Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Dimension-Free Approximation of Deep Neural Networks for Symmetric Korobov Functions

Yulong Lu, Tong Mao, Jinchao Xu, Yahong Yang

TL;DR

This work proves that symmetric Deep Neural Networks can approximate symmetric Korobov functions without suffering from the curse of dimensionality. By leveraging energy-based sparse-grid representations and permutation symmetry, the authors construct symmetric squared ReLU networks that achieve a dimension-free $H^1$-approximation rate of $O(m^{-1})$ for $f\in X^{2,2}_{\mathrm{sym}}(\Omega)$, with network width and parameter counts that scale only polynomially in the ambient dimension $d$. They further establish nearly optimal generalization bounds for learning gradient fields of symmetric Korobov functions from finite samples, with a rate $\mathbb{E}\|f_{\mathcal{S},\mathcal{F}_{m,L}}-f_{\rho}\|_{E}^{2} \lesssim \left(\frac{(\log M)^2}{M}\right)^{2/3}$ up to polynomial dependence on $d$. The combination of symmetric sparse-grid construction and squared ReLU activations yields practical, theory-backed guarantees for dimension-robust learning of permutation-invariant potentials and related gradient fields. These results advance understanding of symmetry-informed architectures in high-dimensional approximation and provide rigorous foundations for gradient-based learning tasks in physics-informed contexts.

Abstract

Deep neural networks have been widely used as universal approximators for functions with inherent physical structures, including permutation symmetry. In this paper, we construct symmetric deep neural networks to approximate symmetric Korobov functions and prove that both the convergence rate and the constant prefactor scale at most polynomially with respect to the ambient dimension. This represents a substantial improvement over prior approximation guarantees that suffer from the curse of dimensionality. Building on these approximation bounds, we further derive a generalization-error rate for learning symmetric Korobov functions whose leading factors likewise avoid the curse of dimensionality.

On the Dimension-Free Approximation of Deep Neural Networks for Symmetric Korobov Functions

TL;DR

This work proves that symmetric Deep Neural Networks can approximate symmetric Korobov functions without suffering from the curse of dimensionality. By leveraging energy-based sparse-grid representations and permutation symmetry, the authors construct symmetric squared ReLU networks that achieve a dimension-free -approximation rate of for , with network width and parameter counts that scale only polynomially in the ambient dimension . They further establish nearly optimal generalization bounds for learning gradient fields of symmetric Korobov functions from finite samples, with a rate up to polynomial dependence on . The combination of symmetric sparse-grid construction and squared ReLU activations yields practical, theory-backed guarantees for dimension-robust learning of permutation-invariant potentials and related gradient fields. These results advance understanding of symmetry-informed architectures in high-dimensional approximation and provide rigorous foundations for gradient-based learning tasks in physics-informed contexts.

Abstract

Deep neural networks have been widely used as universal approximators for functions with inherent physical structures, including permutation symmetry. In this paper, we construct symmetric deep neural networks to approximate symmetric Korobov functions and prove that both the convergence rate and the constant prefactor scale at most polynomially with respect to the ambient dimension. This represents a substantial improvement over prior approximation guarantees that suffer from the curse of dimensionality. Building on these approximation bounds, we further derive a generalization-error rate for learning symmetric Korobov functions whose leading factors likewise avoid the curse of dimensionality.

Paper Structure

This paper contains 14 sections, 19 theorems, 136 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Notations
  3. Structure of the paper
  4. Set-up and main results
  5. Approximation
  6. Generalization
  7. Sparse grid approximation
  8. Symmetric sparse grid approximation for symmetric Korobov functions
  9. Proof of Theorem \ref{['squarebetter']}
  10. Approximating Symmetric Basis Functions with Squared ReLU Networks
  11. Proof of Theorem \ref{['squarebetter']}
  12. Proof of Theorem \ref{['general thm']}
  13. Acknowledgment
  14. Proof of Proposition \ref{['prop:pdim']}

Key Result

Theorem 1

Let $f\in X^{2,2}_{\mathrm{sym}}(\Omega)$ and $n\in\mathbb N_{+}$. There exists a symmetric squared ReLU network $\phi_{\mathrm{ReLU}_2}:\Omega\to\mathbb R$ whose width equals $C_{s}\,6d^{3}\bigl(2^{d-1}-1\bigr)2^{n}\mathrm{e}^{\pi\sqrt{5d/3}},$ whose depth equals $\lfloor\log_2d\rfloor+2$, and whos such that with $\phi_{\mathrm{ReLU}_2}|_{\partial\Omega}=0$, where $\|\cdot\|_E$ is the energy nor

Figures (3)

  • Figure 1: The difference between $\mathbb{X}_n$ and $\mathbb{V}_n$ for $n=5$ and $d=2$.
  • Figure 2: structure of $\prod_{s=1}^d x_s$
  • Figure 3: Approximation of the hat function $h$ by $h_\delta$ with a smaller support.

Theorems & Definitions (38)

  • Definition 1: Korobov Space bungartz2004sparsekorobov2000coulomb
  • Theorem 1
  • Remark 1
  • Remark 2
  • Theorem 2
  • Remark 3
  • Remark 4
  • Lemma 1: bungartz2004sparse, Lemma 3.13
  • Lemma 2: bungartz2004sparse, Theorem 3.10
  • Lemma 3: bungartz2004sparse, Lemma 3.9
  • ...and 28 more