Embedding Inequalities for Barron-type Spaces
Lei Wu
TL;DR
The paper establishes a dimension-free embedding between Barron-type function spaces for compact domains: for $s\in\mathbb{N}^+$ and $\delta\in(0,1)$, the inequality $\delta \|f\|_{\mathcal{F}_{s-\delta}(\Omega)} \lesssim_s \|f\|_{\mathcal{B}_s(\Omega)} \lesssim_s \|f\|_{\mathcal{F}_{s+1}(\Omega)}$ holds, linking the Barron and spectral Barron norms with constants independent of the input dimension. The upper bound aligns with previous results, while the main contribution is proving the sharp, dimension-free lower bound via a Fourier-analytic construction of neuron extensions and a measure-based representation of $f$. The results are shown to be tight, with a concrete example establishing the necessity of $\delta>0$, and they suggest that the embedding is robust in high dimensions. These findings unify Barron-type spaces and have potential impact on high-dimensional approximation of functions and PDEs with two-layer networks, motivating extensions to non-integer $s$ and other activations.
Abstract
An important problem in machine learning theory is to understand the approximation and generalization properties of two-layer neural networks in high dimensions. To this end, researchers have introduced the Barron space $\mathcal{B}_s(Ω)$ and the spectral Barron space $\mathcal{F}_s(Ω)$, where the index $s\in [0,\infty)$ indicates the smoothness of functions within these spaces and $Ω\subset\mathbb{R}^d$ denotes the input domain. However, the precise relationship between the two types of Barron spaces remains unclear. In this paper, we establish a continuous embedding between them as implied by the following inequality: for any $δ\in (0,1), s\in \mathbb{N}^{+}$ and $f: Ω\mapsto\mathbb{R}$, it holds that \[ δ\|f\|_{\mathcal{F}_{s-δ}(Ω)}\lesssim_s \|f\|_{\mathcal{B}_s(Ω)}\lesssim_s \|f\|_{\mathcal{F}_{s+1}(Ω)}. \] Importantly, the constants do not depend on the input dimension $d$, suggesting that the embedding is effective in high dimensions. Moreover, we also show that the lower and upper bound are both tight.