Sharp Lower Bounds for Linearized ReLU^k Approximation on the Sphere

TL;DR

This work proves a sharp saturation lower bound for linearized shallow ReLU$^k$ networks on the sphere $\mathbb{S}^d$. For target functions with Sobolev regularity $r>\frac{d+2k+1}{2}$ and antipodally quasi-uniform centers, the best $\mathcal{L}^2(\mathbb{S}^d)$-approximation cannot improve beyond $n^{-{\frac{d+2k+1}{2d}}}$, matching known upper bounds and yielding the exact saturation order. The authors develop a localized-spherical-polynomial framework, including a decomposition of the network norm into $\sum_q a^\top Q_q a$ and a highly localized Jacobi-based kernel, to prove the lower bound. They show that the apparent advantage of ReLU$^k$ networks over finite elements is fundamentally limited by a regularity barrier, aligning neural-network approximation with classical methods. The results motivate extensions to general domains and nonlinear shallow networks, and clarify the role of antipodal quasi-uniformity in guaranteeing these limits.

Abstract

We prove a saturation theorem for linearized shallow ReLU$^k$ neural networks on the unit sphere $\mathbb S^d$. For any antipodally quasi-uniform set of centers, if the target function has smoothness $r>\tfrac{d+2k+1}{2}$, then the best $\mathcal{L}^2(\mathbb S^d)$ approximation cannot converge faster than order $n^{-\frac{d+2k+1}{2d}}$. This lower bound matches existing upper bounds, thereby establishing the exact saturation order $\tfrac{d+2k+1}{2d}$ for such networks. Our results place linearized neural-network approximation firmly within the classical saturation framework and show that, although ReLU$^k$ networks outperform finite elements under equal degrees $k$, this advantage is intrinsically limited.

Theorems & Definitions (10)

• Definition 2.1: Sobolev spaces on the sphere
• Definition 2.2: Quasi-uniform and antipodally quasi-uniform
• Theorem 2.1
• proof
• Lemma 2.1
• proof
• Theorem 3.1
• proof
• Remark 3.1
• Remark 3.2