Probability error bounds for approximation of functions in reproducing kernel Hilbert spaces
Ata Deniz Aydin, Aurelian Gheondea
TL;DR
This paper addresses how to bound the probability error when approximating functions $f$ in a separable RKHS ${\mathcal{H}}$ by finite sums of kernel translates and by projections onto spans generated by i.i.d. samples from a measure $P$. It introduces the operator $L_{P,K}$ and defines the RKHS ${\mathcal{H}}_P$ as its range, deriving a closed-form kernel $K_P$ and explicit RKHS-norm bounds for approximation errors. A key contribution is proving that, if ${\mathcal{H}}_P$ is dense in ${\mathcal{H}}$, then projections converge almost surely to the identity on a uniqueness set, which implies almost sure convergence for iid samples; the results are illustrated with the uniform distribution on a compact interval and the Hardy space $H^2({\mathbb{D}})$. Overall, the work strengthens prior $L^p$-type bounds by yielding RKHS-norm convergence and upgrading probabilistic convergence to almost sure, within a separable RKHS framework and a Bochner-integral approach.
Abstract
We find probability error bounds for approximations of functions $f$ in a separable reproducing kernel Hilbert space $\mathcal{H}$ with reproducing kernel $K$ on a base space $X$, firstly in terms of finite linear combinations of functions of type $K_{x_i}$ and then in terms of the projection $π^n_x$ on $\mathrm{Span}\{K_{x_i}\}^n_{i=1}$, for random sequences of points $x=(x_i)_i$ in $X$. Given a probability measure $P$, letting $P_K$ be the measure defined by $\mathrm{d} P_K(x)=K(x,x)\mathrm{d} P(x)$, $x\in X$, our approach is based on the nonexpansive operator \[L^2(X;P_K)\niλ\mapsto L_{P,K}λ:=\int_X λ(x)K_x\mathrm{d} P(x)\in \mathcal{H},\] where the integral exists in the Bochner sense. Using this operator, we then define a new reproducing kernel Hilbert space, denoted by $\mathcal{H}_P$, that is the operator range of $L_{P,K}$. Our main result establishes bounds, in terms of the operator $L_{P,K}$, on the probability that the Hilbert space distance between an arbitrary function $f\in\mathcal{H}$ and linear combinations of functions of type $K_{x_i}$, for $(x_i)_i$ sampled independently from $P$, falls below a given threshold. For sequences of points $(x_i)_{i=1}^\infty$ constituting a so-called uniqueness set, the orthogonal projections $π^n_x$ to $\mathrm{Span}\{K_{x_i}\}^n_{i=1}$ converge in the strong operator topology to the identity operator. We prove that, under the assumption that $\mathcal{H}_P$ is dense in $\mathcal{H}$, any sequence of iid samples from $P$ yields a uniqueness set with probability $1$. This result improves on previous error bounds in weaker norms, such as uniform or $L^p$ norms, which yield only convergence in probability and not a.c. convergence. Two examples that show the applicability of this result to a uniform distribution on a compact interval and to the Hardy space $H^2(\mathbb{D})$ are presented as well.