Universal Approximation of Continuous Functionals on Compact Subsets via Linear Measurements and Scalar Nonlinearities
Andrey Krylov, Maksim Penkin
TL;DR
We address the problem of universal approximation for continuous functionals on compact subsets $K$ of product Hilbert spaces. The main approach decouples sensing from learning and shows that any $f\in C(K)$ can be uniformly approximated by a finite sum $g(x)=\sum_{j=1}^r \zeta_j\Bigl(\sum_{i=1}^n \varphi_{ji}(x_i)\Bigr)$, where $\varphi_{ji}\in H^*$ are linear measurements and $\zeta_j$ are continuous scalars. An operator-valued extension yields finite-rank approximations $f:K\to Y$ for Banach $Y$, and a metric-entropy viewpoint provides quantitative proxies via covering numbers. This framework justifies measure-then-nonlinear pipelines in operator learning and imaging, connecting classical inverse problems, Kurmogorov–Arnold networks, and modern neural-operator architectures as instances of the same compact-set universal-approximation principle.
Abstract
We study universal approximation of continuous functionals on compact subsets of products of Hilbert spaces. We prove that any such functional can be uniformly approximated by models that first take finitely many continuous linear measurements of the inputs and then combine these measurements through continuous scalar nonlinearities. We also extend the approximation principle to maps with values in a Banach space, yielding finite-rank approximations. These results provide a compact-set justification for the common ``measure, apply scalar nonlinearities, then combine'' design pattern used in operator learning and imaging.