Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spectral properties of kernel matrices in the flat limit

Simon Barthelmé, Konstantin Usevich

TL;DR

This manuscript focuses on spectral properties of kernel matrices in the so-called "flat limit", which occurs when points are close together relative to the scale of the kernel.

Abstract

Kernel matrices are of central importance to many applied fields. In this manuscript, we focus on spectral properties of kernel matrices in the so-called ``flat limit'', which occurs when points are close together relative to the scale of the kernel. We establish asymptotic expressions for the determinants of the kernel matrices, which we then leverage to obtain asymptotic expressions for the main terms of the eigenvalues. Analyticity of the eigenprojectors yields expressions for limiting eigenvectors, which are strongly tied to discrete orthogonal polynomials. Both smooth and finitely smooth kernels are covered, with stronger results available in the finite smoothness case.

Spectral properties of kernel matrices in the flat limit

TL;DR

This manuscript focuses on spectral properties of kernel matrices in the so-called "flat limit", which occurs when points are close together relative to the scale of the kernel.

Abstract

Kernel matrices are of central importance to many applied fields. In this manuscript, we focus on spectral properties of kernel matrices in the so-called ``flat limit'', which occurs when points are close together relative to the scale of the kernel. We establish asymptotic expressions for the determinants of the kernel matrices, which we then leverage to obtain asymptotic expressions for the main terms of the eigenvalues. Analyticity of the eigenprojectors yields expressions for limiting eigenvectors, which are strongly tied to discrete orthogonal polynomials. Both smooth and finitely smooth kernels are covered, with stronger results available in the finite smoothness case.

Paper Structure

This paper contains 42 sections, 25 theorems, 200 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Overview of the results
  3. Types of kernels
  4. Univariate results
  5. The multivariate case
  6. Overview of tools used in the paper
  7. Organisation of the paper
  8. Background and main notation
  9. Translation invariant and radial kernels
  10. Distance matrices, Vandermonde matrices, and their properties
  11. Scaling and expansions of kernel matrices
  12. Determinants and elementary symmetric polynomials
  13. Eigenvalues, principal minors and elementary symmetric polynomials
  14. Orders of elementary symmetric polynomials
  15. Orders of eigenvalues and eigenvectors
  16. ...and 27 more sections

Key Result

Lemma 2.1

For a distinct node set $\mathcal{X} \subset \mathbb{R}$ of size $n>r \ge 1$, we let $\boldsymbol{B}$ be a full column rank matrix such that $\boldsymbol{B}^{{\sf T}}\boldsymbol{V}_{\le r-1} = 0$. Then the matrix $(-1)^r \boldsymbol{B}^{{\sf T}} \boldsymbol{D}_{(2r-1)} \boldsymbol{B}$ is positive de

Figures (8)

  • Figure 1: Eigenvalues of the Gaussian kernel ($d=1$). The set of $n=10$ nodes was drawn uniformly from the unit interval. In black, eigenvalues of the Gaussian kernel, for different values of $\varepsilon$. The dashed red curves are our small-$\varepsilon$ expansions. Note that both axes are scaled logarithmically. The noise apparent for small $\varepsilon$ values in the low range is due to loss of precision in the numerical computations.
  • Figure 1: Sets of multiindices, $d=2$. Black dots: $\mathbb{P}_2$, grey dots: $\mathbb{H}_3$.
  • Figure 1: Classes of staircase matrices. White color stands for zero blocks.
  • Figure 2: Eigenvalues of the exponential kernel ($d=1$), using the same set of points as in \ref{['fig:eval-gaussian-kernel-1d']}. The largest eigenvalue has a slope of 0 for small $\varepsilon$, the others have unit slope, as in \ref{['thm:det_1d_finite_smoothness']}.
  • Figure 3: First four eigenvectors of the Gaussian kernel. We used the same set of points as in the previous two figures. In blue, eigenvectors of the Gaussian kernel, for different values of $\varepsilon$. The dashed red curve shows the theoretical limit as $\varepsilon \rightarrow 0$ (i.e., the first four orthogonal polynomials of the discrete measure)
  • ...and 3 more figures

Theorems & Definitions (57)

  • Lemma 2.1
  • Theorem 3.1: horn1990matrix
  • Remark 3.2
  • Lemma 3.3
  • Proof 1
  • Lemma 3.4
  • Lemma 3.5
  • Remark 3.6
  • Theorem 3.7: horn1990matrix
  • Lemma 3.8
  • ...and 47 more