Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Approximation in Hilbert spaces of the Gaussian and related analytic kernels

Toni Karvonen, Yuya Suzuki

TL;DR

This work analyzes linear approximation in RKHSs generated by analytic kernels on $[-1,1]$, with a focus on the Gaussian kernel and related weighted power-series and stationary kernels. It develops a unified framework that yields lower and upper bounds on worst-case errors for standard information, showing that the $n$-th minimal error decays essentially like $\alpha_n^{-1/2}$ up to sub-exponential factors, where $\alpha_n$ encodes kernel smoothness (e.g., for the Gaussian, $\alpha_n=\varepsilon^{-2n}n!$). The main contributions include (i) a general bound theory for weighted power-series kernels via weighted polynomial interpolation and polynomial-coefficient estimates, (ii) analogous results for stationary kernels with Fourier-domain quantities $\beta_k$, and (iii) concrete sharp bounds for the Gaussian kernel, including derivative-information cases and higher-dimensional remarks, complemented by a comparison with Yarotsky's posterior-variance bounds. These results offer precise decay rates tied to kernel shape parameters and have implications for covariance-function parameter estimation and kriging in Gaussian-process modelling. They also clarify the limitations of extending these bounds to higher dimensions and highlight the role of interpolation strategy in attaining tight guarantees.

Abstract

We consider linear approximation based on function evaluations in reproducing kernel Hilbert spaces of certain analytic weighted power series kernels and stationary kernels on the interval $[-1,1]$. Both classes contain the popular Gaussian kernel $K(x, y) = \exp(-\tfrac{1}{2}\varepsilon^2(x-y)^2)$. For weighted power series kernels we derive almost matching upper and lower bounds on the worst-case error. When applied to the Gaussian kernel, our results state that, up to a sub-exponential factor, the $n$th minimal error decays as $(\varepsilon/2)^n (n!)^{-1/2}$. The proofs are based on weighted polynomial interpolation and classical polynomial coefficient estimates that we use to bound the Hilbert space norm of a weighted polynomial fooling function.

Approximation in Hilbert spaces of the Gaussian and related analytic kernels

TL;DR

This work analyzes linear approximation in RKHSs generated by analytic kernels on , with a focus on the Gaussian kernel and related weighted power-series and stationary kernels. It develops a unified framework that yields lower and upper bounds on worst-case errors for standard information, showing that the -th minimal error decays essentially like up to sub-exponential factors, where encodes kernel smoothness (e.g., for the Gaussian, ). The main contributions include (i) a general bound theory for weighted power-series kernels via weighted polynomial interpolation and polynomial-coefficient estimates, (ii) analogous results for stationary kernels with Fourier-domain quantities , and (iii) concrete sharp bounds for the Gaussian kernel, including derivative-information cases and higher-dimensional remarks, complemented by a comparison with Yarotsky's posterior-variance bounds. These results offer precise decay rates tied to kernel shape parameters and have implications for covariance-function parameter estimation and kriging in Gaussian-process modelling. They also clarify the limitations of extending these bounds to higher dimensions and highlight the role of interpolation strategy in attaining tight guarantees.

Abstract

We consider linear approximation based on function evaluations in reproducing kernel Hilbert spaces of certain analytic weighted power series kernels and stationary kernels on the interval . Both classes contain the popular Gaussian kernel . For weighted power series kernels we derive almost matching upper and lower bounds on the worst-case error. When applied to the Gaussian kernel, our results state that, up to a sub-exponential factor, the th minimal error decays as . The proofs are based on weighted polynomial interpolation and classical polynomial coefficient estimates that we use to bound the Hilbert space norm of a weighted polynomial fooling function.
Paper Structure (12 sections, 22 theorems, 154 equations)

This paper contains 12 sections, 22 theorems, 154 equations.

Table of Contents

  1. Introduction
  2. Prelude: Korobov spaces
  3. Weighted power series kernels
  4. Lower bounds for approximation in $L^\infty([-1, 1])$
  5. Lower bounds for approximation in $L^2([-1, 1])$
  6. Upper bounds
  7. Application to specific kernels
  8. Derivative information
  9. Some remarks on higher dimensions
  10. Stationary kernels
  11. Proof of \ref{['thm:main-4']}
  12. Bounds by Yarotsky

Key Result

theorem 1.1

Let $K(x, y) = \exp(-\tfrac{1}{2} \varepsilon^2 (x - y)^2)$ be the Gaussian kernel, $p \in \{2, \infty\}$, and $c_L$ and $m_L$ the functions defined in eq:cL-constant and eq:mL-constant. If $x_1, \ldots, x_n \in [-1, 1]$ are distinct, then where the lower bounds hold when $n \geq 1$ and the upper bounds when $n \geq m_L(\varepsilon^2)$, the constants $c_{1, \infty}$ and $c_{1,2}$ are obtained by

Theorems & Definitions (42)

  • theorem 1.1
  • Proposition 2.1
  • proof
  • theorem 3.1
  • lemma 3.2
  • proof
  • remark 3.3
  • lemma 3.4
  • proof
  • lemma 3.5
  • ...and 32 more