Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Weighted Sobolev Approximation Rates for Neural Networks on Unbounded Domains

Ahmed Abdeljawad, Thomas Dittrich

TL;DR

This work establishes asymptotic approximation rates for shallow neural networks that come without curse of dimensionality and presents embedding results for the more general weighted Fourier-Lebesgue spaces in the weighted Sobolev spaces.

Abstract

In this work, we consider the approximation capabilities of shallow neural networks in weighted Sobolev spaces for functions in the spectral Barron space. The existing literature already covers several cases, in which the spectral Barron space can be approximated well, i.e., without curse of dimensionality, by shallow networks and several different classes of activation function. The limitations of the existing results are mostly on the error measures that were considered, in which the results are restricted to Sobolev spaces over a bounded domain. We will here treat two cases that extend upon the existing results. Namely, we treat the case with bounded domain and Muckenhoupt weights and the case, where the domain is allowed to be unbounded and the weights are required to decay. We first present embedding results for the more general weighted Fourier-Lebesgue spaces in the weighted Sobolev spaces and then we establish asymptotic approximation rates for shallow neural networks that come without curse of dimensionality.

Weighted Sobolev Approximation Rates for Neural Networks on Unbounded Domains

TL;DR

This work establishes asymptotic approximation rates for shallow neural networks that come without curse of dimensionality and presents embedding results for the more general weighted Fourier-Lebesgue spaces in the weighted Sobolev spaces.

Abstract

In this work, we consider the approximation capabilities of shallow neural networks in weighted Sobolev spaces for functions in the spectral Barron space. The existing literature already covers several cases, in which the spectral Barron space can be approximated well, i.e., without curse of dimensionality, by shallow networks and several different classes of activation function. The limitations of the existing results are mostly on the error measures that were considered, in which the results are restricted to Sobolev spaces over a bounded domain. We will here treat two cases that extend upon the existing results. Namely, we treat the case with bounded domain and Muckenhoupt weights and the case, where the domain is allowed to be unbounded and the weights are required to decay. We first present embedding results for the more general weighted Fourier-Lebesgue spaces in the weighted Sobolev spaces and then we establish asymptotic approximation rates for shallow neural networks that come without curse of dimensionality.

Paper Structure

This paper contains 12 sections, 23 theorems, 115 equations.

Table of Contents

  1. Introduction
  2. Contributions and Discussion
  3. Preliminaries
  4. Weighted Function Spaces: Definition and Properties
  5. Smoothing by Convolution
  6. Maurey Approximation
  7. Embedding Results for Fourier-Lebesgue Spaces
  8. Embedding in Sobolev Spaces with p >= 2
  9. Embedding in Sobolev Spaces with p in [1,2]
  10. Approximation of Fourier-Lebesgue Spaces
  11. Error Measure over Bounded Domain
  12. Error Measure over Unbounded Domain

Key Result

Corollary 1

Let $d,\ell\in\mathbb{N}^{}$, $\gamma\in\mathbb{R}^{}$ with $\gamma> d/2$, $p\in[2,\infty]$, $q=2(p/2)^\prime$, $\mathcal{U}\subset\mathbb{R}^{d}$ have finite volume, and where $\upsilon$ is a radial non-decreasing function such that $\upsilon\in A_{p^\prime}(\mathbb{R}^{d})$ with $\upsilon(x)\geq \left\langle {1/ \left| {x}\right| }\right\rangle ^{-\gamma p^\prime}$. Then for any $f\in \mathcal{

Theorems & Definitions (32)

  • Corollary 1
  • Lemma 2: Embedding of Barron Space
  • Theorem 3: Approximation in weighted Sobolev Space
  • Theorem 4
  • Definition 5: Muckenhoupt Weights Heinig89FourierInequalitiesIntegral
  • Definition 6: Weighted Function Spaces
  • Definition 7: Spectral Barron Space
  • Remark 8: Simplified Notation
  • Proposition 9: Higher-Order Embedding Abdeljawad23SpaceTimeApproximationShallow
  • Lemma 10: Generalized Hausdorff-Young Inequality: Type I Heinig89FourierInequalitiesIntegral
  • ...and 22 more