Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Numerical Methods for Kernel Slicing

Nicolaj Rux, Johannes Hertrich, Sebastian Neumayer

TL;DR

This work tackles the computational bottleneck of kernel sums by employing kernel slicing to rewrite $F(\|x\|)$ as $\mathcal{S}_d[f](\|x\|)$, thereby reducing multidimensional sums to $P$ one-dimensional problems along random directions. It recasts the recovery of the radial slicer $f$ as an inverse problem and develops two regularized cosine-coefficient reconstruction strategies, with rigorous error estimates and extensive numerical validation. The authors establish inversion techniques (power-series, Fourier-radial, and derivative-based) for $\mathcal{S}_d$, analyze operator norms, and compare spatial- and frequency-domain minimization schemes, providing practical guidance on selecting function spaces and regularization parameters. The resulting framework yields fast kernel summations with provable error control, achieving significant runtime savings over brute-force methods, especially in high dimensions, making it attractive for large-scale kernel-based applications. Overall, the paper provides a principled and scalable approach to accelerated kernel computations via inverse-slicing and regularized reconstruction of the slicing function.

Abstract

Kernels are key in machine learning for modeling interactions. Unfortunately, brute-force computation of the related kernel sums scales quadratically with the number of samples. Recent Fourier-slicing methods lead to an improved linear complexity, provided that the kernel can be sliced and its Fourier coefficients are known. To obtain these coefficients, we view the slicing relation as an inverse problem and present two algorithms for their recovery. Extensive numerical experiments demonstrate the speed and accuracy of our methods.

Numerical Methods for Kernel Slicing

TL;DR

This work tackles the computational bottleneck of kernel sums by employing kernel slicing to rewrite as , thereby reducing multidimensional sums to one-dimensional problems along random directions. It recasts the recovery of the radial slicer as an inverse problem and develops two regularized cosine-coefficient reconstruction strategies, with rigorous error estimates and extensive numerical validation. The authors establish inversion techniques (power-series, Fourier-radial, and derivative-based) for , analyze operator norms, and compare spatial- and frequency-domain minimization schemes, providing practical guidance on selecting function spaces and regularization parameters. The resulting framework yields fast kernel summations with provable error control, achieving significant runtime savings over brute-force methods, especially in high dimensions, making it attractive for large-scale kernel-based applications. Overall, the paper provides a principled and scalable approach to accelerated kernel computations via inverse-slicing and regularized reconstruction of the slicing function.

Abstract

Kernels are key in machine learning for modeling interactions. Unfortunately, brute-force computation of the related kernel sums scales quadratically with the number of samples. Recent Fourier-slicing methods lead to an improved linear complexity, provided that the kernel can be sliced and its Fourier coefficients are known. To obtain these coefficients, we view the slicing relation as an inverse problem and present two algorithms for their recovery. Extensive numerical experiments demonstrate the speed and accuracy of our methods.

Paper Structure

This paper contains 38 sections, 12 theorems, 84 equations, 7 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contribution and Outline
  3. Background
  4. Related Literature
  5. Slicing
  6. Fast Kernel Summations
  7. Fast Kernel Summations
  8. Fast Fourier Summation in 1D
  9. Notations
  10. The Slicing Operator
  11. Explicit Inversion
  12. Inversion Method 1: Eigendecomposition for Power Series
  13. Inversion Method 2: Fourier Transforms of Radial Basis Functions
  14. Inversion Method 3: Derivative Formula
  15. Operator Norm
  16. ...and 23 more sections

Key Result

Proposition 3

For $c>0$, $d \geq 3$ and $F(s) =(c^2+s^2)^{-1/2}$, it holds that $F = \mathcal{S}_d[f]$ with $f(t)=c^{d-1 }(c^2+t^2)^{-d/2}$ and $\mathcal{F}_1(f) (r)= \omega_{d-1} (c|r|)^{(d-1)/2} K_{(d-1)/2} (2\pi c|r|)$.

Figures (7)

  • Figure 1: Densities $\varrho_d$ and normalization constant $c_d$ appearing for $\mathcal{S}_d$ in \ref{['eq:RLFI']}.
  • Figure 2: The function $\eta_d$ for $d\in \{3,10,100,1000\}$ on $x\in [-200,200]$.
  • Figure 3: Graphs of the (kernel) basis functions $F$ in Table \ref{['tab:testfunc']}.
  • Figure 4: Forward error for the methods of Table \ref{['tab:methods']} in dimension $d=1000$.
  • Figure 5: Kernel Summation: Relative $L^2$-error $\|s-\tilde{s}\|_2/\|s\|_2$ for $N=M=1.0e4$ samples with $P=1000$ slicing directions in dimension $d=1000$.
  • ...and 2 more figures

Theorems & Definitions (25)

  • Remark 1: Relevant Domain
  • Remark 2
  • Proposition 3
  • Proposition 4: Inversion of $\mathcal{S}_d$
  • Theorem 5
  • Example 6
  • Proposition 7
  • Remark 8
  • Proposition 9
  • proof
  • ...and 15 more