Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Non-asymptotic spectral bounds on the $\varepsilon$-entropy of kernel classes

Rustem Takhanov

TL;DR

The paper addresses non-asymptotic bounds on the ε-entropy of the unit ball B_{H_K} in an RKHS H_K for a continuous Mercer kernel on a compact set, quantified in L_p(ν). It develops a spectral approach by viewing B_{H_K} as an infinite-dimensional ellipsoid with axis lengths σ_i = √λ_i and eigenfunctions φ_i, and derives sharp upper and lower bounds that depend on the eigenvalues and, for p > 2, the tail behavior of the Mercer expansion via C_{K,p}(N). The main contribution is a unified framework that yields explicit bounds for all p ∈ [1,∞], including p ≤ 2 where the tail is negligible and p > 2 where the tail matters, with corollaries showing tightness for zonal and Gaussian kernels. The results substantially tighten previous worst-case bounds and highlight the role of intrinsic dimension through the spectral data, offering practical implications for kernel ridge regression, SVMs, and related kernel methods in non-asymptotic regimes.

Abstract

Let $K: \boldsymbolΩ\times \boldsymbolΩ$ be a continuous Mercer kernel defined on a compact subset of ${\mathbb R}^n$ and $\mathcal{H}_K$ be the reproducing kernel Hilbert space (RKHS) associated with $K$. Given a finite measure $ν$ on $\boldsymbolΩ$, we investigate upper and lower bounds on the $\varepsilon$-entropy of the unit ball of $\mathcal{H}_K$ in the space $L_p(ν)$. This topic is an important direction in the modern statistical theory of kernel-based methods. We prove sharp upper and lower bounds for $p\in [1,+\infty]$. For $p\in [1,2]$, the upper bounds are determined solely by the eigenvalue behaviour of the corresponding integral operator $φ\to \int_{\boldsymbolΩ} K(\cdot,{\mathbf y})φ({\mathbf y})dν({\mathbf y})$. In constrast, for $p>2$, the bounds additionally depend on the convergence rate of the truncated Mercer series to the kernel $K$ in the $L_p(ν)$-norm. We discuss a number of consequences of our bounds and show that they are substantially tighter than previous bounds for general kernels. Furthermore, for specific cases, such as zonal kernels and the Gaussian kernel on a box, our bounds are asymptotically tight as $\varepsilon\to +0$.

Non-asymptotic spectral bounds on the $\varepsilon$-entropy of kernel classes

TL;DR

The paper addresses non-asymptotic bounds on the ε-entropy of the unit ball B_{H_K} in an RKHS H_K for a continuous Mercer kernel on a compact set, quantified in L_p(ν). It develops a spectral approach by viewing B_{H_K} as an infinite-dimensional ellipsoid with axis lengths σ_i = √λ_i and eigenfunctions φ_i, and derives sharp upper and lower bounds that depend on the eigenvalues and, for p > 2, the tail behavior of the Mercer expansion via C_{K,p}(N). The main contribution is a unified framework that yields explicit bounds for all p ∈ [1,∞], including p ≤ 2 where the tail is negligible and p > 2 where the tail matters, with corollaries showing tightness for zonal and Gaussian kernels. The results substantially tighten previous worst-case bounds and highlight the role of intrinsic dimension through the spectral data, offering practical implications for kernel ridge regression, SVMs, and related kernel methods in non-asymptotic regimes.

Abstract

Let be a continuous Mercer kernel defined on a compact subset of and be the reproducing kernel Hilbert space (RKHS) associated with . Given a finite measure on , we investigate upper and lower bounds on the -entropy of the unit ball of in the space . This topic is an important direction in the modern statistical theory of kernel-based methods. We prove sharp upper and lower bounds for . For , the upper bounds are determined solely by the eigenvalue behaviour of the corresponding integral operator . In constrast, for , the bounds additionally depend on the convergence rate of the truncated Mercer series to the kernel in the -norm. We discuss a number of consequences of our bounds and show that they are substantially tighter than previous bounds for general kernels. Furthermore, for specific cases, such as zonal kernels and the Gaussian kernel on a box, our bounds are asymptotically tight as .
Paper Structure (12 sections, 30 theorems, 209 equations)

This paper contains 12 sections, 30 theorems, 209 equations.

Table of Contents

  1. Introduction and main results
  2. $\varepsilon$-entropy of a ball in $\mathcal{H}_{K}$
  3. Main results
  4. Upper bounds on the $\varepsilon$-entropy of the main part of $B_{{\mathcal{H}}_K}$
  5. Upper bounds on the $\varepsilon$-entropy of the tail part of $B_{{\mathcal{H}}_K}$
  6. Proof of the main upper bound
  7. Bounding the decay rate of $C_{K,\infty}(N)$
  8. Lower bounds based on volume considerations
  9. Growth rates of $\mathcal{E}$ and $m_\varepsilon$
  10. Twice differentiable kernels and the property \ref{['property-k']}
  11. Application I: zonal kernels
  12. Application II: the Gaussian kernel on a box

Key Result

Proposition 1

Let $\nu$ be a Borel, nondegenerate measure on $\boldsymbol{\Omega}$. Let $\{\lambda_i\}_{i=1}^\infty$ be the set of all positive eigenvalues of ${\rm O}_K$ (counting multiplicities) with corresponding orthogonal unit eigenvectors $\{\phi_i\}_{i=1}^\infty$. Then, the RKHS of $K$, denoted by $\mathca with the inner product $\langle \sum_{i=1}^\infty a_i\phi_i , \sum_{i=1}^\infty b_i\phi_i \rangle_{

Theorems & Definitions (66)

  • Proposition 1: Cucker2001OnTMcucker_zhou_2007
  • Definition 1
  • Theorem 2
  • Corollary 1
  • Remark 1
  • Remark 2
  • Remark 3
  • Definition 2
  • Corollary 2
  • Theorem 3
  • ...and 56 more