Blessing of Dimensionality for Approximating Sobolev Classes on Manifolds
Hong Ye Tan, Subhadip Mukherjee, Junqi Tang, Carola-Bibiane Schönlieb
TL;DR
This work proves ambient-dimension-free lower bounds for approximating bounded Sobolev function classes on compact Riemannian manifolds by finite-pseudo-dimension function classes. The main result shows ρ_n(W^{1,p}(1), L^q(M)) ≥ C(d,g,K,vol(M),p,q) (n + log n)^{-1/d}, with constants depending only on intrinsic geometric quantities such as curvature, volume, and injectivity radius, not on any ambient embedding. The authors develop a geometric packing-based construction and leverage Bishop–Gromov volume comparison and metric-entropy arguments to establish the lower bound, while situating it alongside existing upper-bound results for ReLU networks and Hölder approximations on manifolds. Practically, this implies a fundamental, ambient-dimension-free limit on generalization and approximation for manifold-supported data, guiding expectations for learning models operating under the manifold hypothesis.
Abstract
The manifold hypothesis says that natural high-dimensional data lie on or around a low-dimensional manifold. The recent success of statistical and learning-based methods in very high dimensions empirically supports this hypothesis, suggesting that typical worst-case analysis does not provide practical guarantees. A natural step for analysis is thus to assume the manifold hypothesis and derive bounds that are independent of any ambient dimensions that the data may be embedded in. Theoretical implications in this direction have recently been explored in terms of generalization of ReLU networks and convergence of Langevin methods. In this work, we consider optimal uniform approximations with functions of finite statistical complexity. While upper bounds on uniform approximation exist in the literature using ReLU neural networks, we consider the opposite: lower bounds to quantify the fundamental difficulty of approximation on manifolds. In particular, we demonstrate that the statistical complexity required to approximate a class of bounded Sobolev functions on a compact manifold is bounded from below, and moreover that this bound is dependent only on the intrinsic properties of the manifold, such as curvature, volume, and injectivity radius.