Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Approximation of Kernel functions

Paul Dommel, Alois Pichler

TL;DR

This work develops an explicit kernel approximation framework in RKHS by projecting the kernel onto the range of the Hilbert–Schmidt operator and matching the initial Taylor coefficients of radial kernels. It introduces the minimal moment function $w_m^x$ as the $L^2$-minimizer under moment constraints, provides its explicit polynomial representation via the Hilbert matrix, and extends to high dimensions with a product construction that preserves moment properties. For the Gaussian kernel on compact domains, the authors derive uniform and RKHS-norm bounds for the point-evaluation approximation, show polynomial (rather than exponential) growth of eigenfunctions, and prove an interpolation inequality that bounds uniform error by multivariate $L^2$ error. These results yield improved concentration bounds and justify smaller regularization parameters, enabling more stable low-rank methods such as Nyström with explicit guidance on support-point counts. Overall, the paper connects kernel approximation, eigenstructure, and low-rank kernel methods to enhance accuracy and efficiency in RKHS-based learning on bounded domains.

Abstract

Various methods in statistical learning build on kernels considered in reproducing kernel Hilbert spaces. In applications, the kernel is often selected based on characteristics of the problem and the data. This kernel is then employed to infer response variables at points, where no explanatory data were observed. The data considered here are located in compact sets in higher dimensions and the paper addresses approximations of the kernel itself. The new approach considers Taylor series approximations of radial kernel functions. For the Gauss kernel on the unit cube, the paper establishes an upper bound of the associated eigenfunctions, which grows only polynomially with respect to the index. The novel approach substantiates smaller regularization parameters than considered in the literature, overall leading to better approximations. This improvement confirms low rank approximation methods such as the Nyström method.

On the Approximation of Kernel functions

TL;DR

This work develops an explicit kernel approximation framework in RKHS by projecting the kernel onto the range of the Hilbert–Schmidt operator and matching the initial Taylor coefficients of radial kernels. It introduces the minimal moment function as the -minimizer under moment constraints, provides its explicit polynomial representation via the Hilbert matrix, and extends to high dimensions with a product construction that preserves moment properties. For the Gaussian kernel on compact domains, the authors derive uniform and RKHS-norm bounds for the point-evaluation approximation, show polynomial (rather than exponential) growth of eigenfunctions, and prove an interpolation inequality that bounds uniform error by multivariate error. These results yield improved concentration bounds and justify smaller regularization parameters, enabling more stable low-rank methods such as Nyström with explicit guidance on support-point counts. Overall, the paper connects kernel approximation, eigenstructure, and low-rank kernel methods to enhance accuracy and efficiency in RKHS-based learning on bounded domains.

Abstract

Various methods in statistical learning build on kernels considered in reproducing kernel Hilbert spaces. In applications, the kernel is often selected based on characteristics of the problem and the data. This kernel is then employed to infer response variables at points, where no explanatory data were observed. The data considered here are located in compact sets in higher dimensions and the paper addresses approximations of the kernel itself. The new approach considers Taylor series approximations of radial kernel functions. For the Gauss kernel on the unit cube, the paper establishes an upper bound of the associated eigenfunctions, which grows only polynomially with respect to the index. The novel approach substantiates smaller regularization parameters than considered in the literature, overall leading to better approximations. This improvement confirms low rank approximation methods such as the Nyström method.
Paper Structure (20 sections, 24 theorems, 155 equations)

This paper contains 20 sections, 24 theorems, 155 equations.

Table of Contents

  1. Introduction
  2. Related literature.
  3. Outline of the paper.
  4. The minimal moment function
  5. The minimal moment function
  6. Extensions and error estimates
  7. The Gaussian kernel
  8. Approximation of the point evaluation function
  9. Implications for the weight function
  10. Eigenvalues and eigenfunctions
  11. Main results
  12. Concentration bounds
  13. Interpolation inequality
  14. Convergence in $L^{2}$ implies convergence in $L^{\infty}$:
  15. Nyström method
  16. ...and 5 more sections

Key Result

Theorem 2.1

For $x\in[0,1]$ fixed, the optimization problem has the unique solution where $\alpha_{x}$ satisfies the equations $H_{m}\alpha_{x}=\bar{x}$ for the Hilbert matrix $H_{m}\coloneqq\left(\frac{1}{i+j-1}\right)_{i=1,j=1}^{n,n}$ and the vector $\bar{x}\coloneqq(1,x,\dots,x^{m-1})\in\mathbb{R}^{m}$.

Theorems & Definitions (52)

  • Theorem 2.1: Explicit minimal moment function
  • proof
  • Theorem 2.2: Upper bound of the weight function
  • proof
  • Proposition 2.3: Upper bound of the weight function in higher dimensions
  • proof
  • Remark 2.4
  • Theorem 2.5: Uniform bound in $d$-dimensions
  • proof
  • Theorem 3.1: Uniform approximation of $k_{x}$ in $\left\Vert \cdot\right\Vert _{\infty}$
  • ...and 42 more