Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Low-rank approximation of analytic kernels

Marcus Webb

TL;DR

This work develops a rigorous framework for bounding the best low-rank approximation error of matrices and operators arising from analytic kernels by exploiting a factorization $\mathcal{K}=\mathcal{K}'\,\mathcal{C}$ into a Grothendieck dual and a Cauchy transform. The main contribution is the introduction of Cauchy--Zolotarev numbers $Z_n(\cdot,\cdot)$, which quantify how fast the best rank-$n$ approximation decays, and the demonstration that the optimal low-rank kernel $K_n$ is computable via rational interpolation whose nodes and poles come from the roots and poles of the optimal $\phi$. The paper provides concrete strategies to construct $\phi$, bounds the approximation error in terms of $Z_n$, and validates the theory with numerous examples including Cauchy matrices/tensors, log-Cauchy kernels, twisted Hankel transforms, and Beta-Cauchy kernels. The results offer a principled path to fast, provably accurate low-rank approximations in scientific computing and data analysis, with potential implications for potential theory and related areas due to the Grothendieck duality viewpoint.

Abstract

Many algorithms in scientific computing and data science take advantage of low-rank approximation of matrices and kernels, and understanding why nearly-low-rank structure occurs is essential for their analysis and further development. This paper provides a framework for bounding the best low-rank approximation error of matrices arising from samples of a kernel that is analytically continuable in one of its variables to an open region of the complex plane. Elegantly, the low-rank approximations used in the proof are computable by rational interpolation using the roots and poles of Zolotarev rational functions, leading to a fast algorithm for their construction.

Low-rank approximation of analytic kernels

TL;DR

This work develops a rigorous framework for bounding the best low-rank approximation error of matrices and operators arising from analytic kernels by exploiting a factorization into a Grothendieck dual and a Cauchy transform. The main contribution is the introduction of Cauchy--Zolotarev numbers , which quantify how fast the best rank- approximation decays, and the demonstration that the optimal low-rank kernel is computable via rational interpolation whose nodes and poles come from the roots and poles of the optimal . The paper provides concrete strategies to construct , bounds the approximation error in terms of , and validates the theory with numerous examples including Cauchy matrices/tensors, log-Cauchy kernels, twisted Hankel transforms, and Beta-Cauchy kernels. The results offer a principled path to fast, provably accurate low-rank approximations in scientific computing and data analysis, with potential implications for potential theory and related areas due to the Grothendieck duality viewpoint.

Abstract

Many algorithms in scientific computing and data science take advantage of low-rank approximation of matrices and kernels, and understanding why nearly-low-rank structure occurs is essential for their analysis and further development. This paper provides a framework for bounding the best low-rank approximation error of matrices arising from samples of a kernel that is analytically continuable in one of its variables to an open region of the complex plane. Elegantly, the low-rank approximations used in the proof are computable by rational interpolation using the roots and poles of Zolotarev rational functions, leading to a fast algorithm for their construction.

Paper Structure

This paper contains 13 sections, 12 theorems, 75 equations, 5 figures.

Table of Contents

  1. Proof of the Main Theorem
  2. Cauchy--Zolotarev numbers
  3. The Grothendieck dual operator
  4. Examples
  5. Grothendieck duality
  6. Computation by rational interpolation
  7. Examples
  8. Cauchy matrices and tensors
  9. Log-Cauchy matrices
  10. Twisted Hankel transform matrix
  11. Beta-Cauchy matrices
  12. Conclusion
  13. Proof of Lemma \ref{['lem:logkernel']}

Key Result

Theorem 1.1

Let $K\in C(D \times E)$ be analytically continuable so that $K \in C(D\times F')$ and for each $x \in D$, $K(x,\cdot)$ is analytic in $F'$. Then for $n = 1,2,3,\ldots$, there exists a rank-$n$ kernel $K_n \in C(D \times E)$ such that where $1 \leq p \leq \infty$, the number $Z_n(L_\mu^2(E),L_\nu^p(F))$ is the Cauchy--Zolotarev number, where $\mathcal{R}_n = \mathcal{R}_{n,n}$ is the set of all

Figures (5)

  • Figure 1: Comparison of the best low-rank approximation error with various bounds and interpolation errors for the matrix in Equation \ref{['eqn:introA']} with $N = 100$. The yellow circles come from an analytically derived Zolotarev rational interpolant based on $[0,N]$ and $[-\infty, -\tfrac{1}{2}]$ (where $K(x,\cdot)$ is singular for $x \in [0,N]$), whereas the purple triangles come from a numerically derived Zolotarev rational interpolant based on $\{0,1,\ldots, N\}$ and $\{-\tfrac{1}{2}, -\tfrac{3}{2}, -\tfrac{5}{2}, \ldots, -\infty\}$ (where $K(i,\cdot)$ is singular for $i \in \{0,1,\ldots,N\}$).
  • Figure 2: A diagram showing the relationship between the sets, contours and points occurring in the proof of Theorem \ref{['thm:main']}. Here $\mathrm{Ind}_{\tilde{\Gamma}}(E) = -1$, $\mathrm{Ind}_{\tilde{\Gamma}}(F) = 0$, $\mathrm{Ind}_{\Gamma}(E) = 0$, $\mathrm{Ind}_{\Gamma}(F) = 1$. Note that $\infty$ could also be located within $F$, discussed in the text.
  • Figure 3: Cauchy matrices and tensors. On the left I used $A\in\mathbb{R}^{N\times N}$ in Equation \ref{['eqn:Cauchyexample1']} with $N = 100$, $D = [1, 70]$, $E = [2,100]$. On the right I used $A \in \mathbb{R}^{N\times N \times N}$ in Equation \ref{['eqn:Cauchexample2']} with $N=50$, $D = [1,70]\times[1,199] \subset \mathbb{C}^2$, $E = [2,100]$. In both plots the sample points are equispaced over their respective intervals. For the purple triangles in the right plot, I numerically calculated the optimal rational function for $Z_n(\{(w_k,x_i)\},\{y_j\})$ and evaluated the associated rational interpolant of $K$.
  • Figure 4: Log-Cauchy matrices. I used $A \in \mathbb{R}^{N\times N}$ in Equation \ref{['eqn:logCauchy']} with $N = 100$, $D = E = [1,N]$ and $\{x_i\}$, $\{y_j\}$ randomly uniformly distributed in $[1,N]$ (and $x_1 = y_1 = 1$, $x_N = y_N = N$). The Chebyshev interpolant (blue dots) uses the modified interpolation points mentioned in Remark \ref{['rem:Cheb']}. The Zolotarev interpolant (yellow circles) uses the $\phi$ in the proof of Lemma \ref{['lem:Z1']} for the nodes and poles, whereas the suboptimal Zolotarev interpolants (cyan plus signs) uses the $\phi$ that is optimal for $Z_n(E,F)$.
  • Figure 5: Twisted Hankel transform matrix. I used $A \in \mathbb{R}^{N\times N}$ in Equation \ref{['eqn:twistedHankelmatrix']} with $N = 100$. The Chebyshev interpolant (blue dots) uses the modified interpolation points mentioned in Remark \ref{['rem:Cheb']}. The Zolotarev interpolant (yellow circles) uses the $\phi$ in the proof of Lemma \ref{['lem:Z1']} for the nodes and poles, whereas the semidiscrete Zolotarev interpolants (purple triangles) use the $\phi$ that is optimal for $Z_n(\{\widehat{\omega}_1,\widehat{\omega}_2,\ldots,\widehat{\omega}_N\},[-\infty,0])$ as described in the main text.

Theorems & Definitions (15)

  • Theorem 1.1
  • Definition 2.1
  • Lemma 2.2
  • Lemma 2.3: cf. beckermann2019bounds
  • Definition 3.1: Cauchy--Zolotarev Numbers
  • Lemma 3.2
  • Theorem 3.3: Beckermann--Townsend beckermann2019bounds
  • Lemma 3.4
  • Lemma 3.5
  • Lemma 4.1
  • ...and 5 more