Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Everything is Vecchia: Unifying low-rank and sparse inverse Cholesky approximations

Eagan Kaminetz, Robert J. Webber

TL;DR

This paper shows how the sum is exactly a Vecchia approximation of the original matrix with an augmented sparsity pattern, indicating that Vecchia approximations subsume a class of existing matrix approximations and have broad applicability.

Abstract

The partial pivoted Cholesky approximation accurately represents matrices that are close to being low-rank. Meanwhile, the Vecchia approximation accurately represents matrices with inverse Cholesky factors that are close to being sparse. What happens if a partial Cholesky approximation is combined with a Vecchia approximation of the residual? This paper shows how the sum is exactly a Vecchia approximation of the original matrix with an augmented sparsity pattern. Thus, Vecchia approximations subsume a class of existing matrix approximations and have broad applicability.

Everything is Vecchia: Unifying low-rank and sparse inverse Cholesky approximations

TL;DR

This paper shows how the sum is exactly a Vecchia approximation of the original matrix with an augmented sparsity pattern, indicating that Vecchia approximations subsume a class of existing matrix approximations and have broad applicability.

Abstract

The partial pivoted Cholesky approximation accurately represents matrices that are close to being low-rank. Meanwhile, the Vecchia approximation accurately represents matrices with inverse Cholesky factors that are close to being sparse. What happens if a partial Cholesky approximation is combined with a Vecchia approximation of the residual? This paper shows how the sum is exactly a Vecchia approximation of the original matrix with an augmented sparsity pattern. Thus, Vecchia approximations subsume a class of existing matrix approximations and have broad applicability.
Paper Structure (34 sections, 9 theorems, 101 equations, 9 figures, 2 tables, 3 algorithms)

This paper contains 34 sections, 9 theorems, 101 equations, 9 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Motivation
  2. Theoretical optimality
  3. Empirical performance
  4. Organization
  5. Notation
  6. Partial Cholesky + Vecchia = Vecchia
  7. Factored approximations based on the Cholesky decomposition
  8. Partial pivoted Cholesky
  9. Vecchia approximation
  10. Partial Cholesky + Vecchia = Vecchia
  11. Implications
  12. Kaporin optimality theory
  13. Vecchia optimality theorem
  14. Linear solves
  15. Determinant calculations
  16. ...and 19 more sections

Key Result

Theorem 2.4

Given a target positive-semidefinite matrix $\bm{A} \in \mathbb{C}^{n \times n}$, consider the following two-part approximation. Then $\hat{\bm{A}}_{\rm part} + \hat{\bm{A}}_{\rm res}$ can be rewritten as a Vecchia approximation of $\bm{A}$ with permutation $\bm{P}$ and an augmented sparsity pattern $\mathsf{S}_i = \bigl(\{1, \ldots, r\} \cup \mathsf{Q}_i\bigr) \cap \{1, \ldots, i-1\}$.

Figures (9)

  • Figure 1: Comparison of five preconditioners based on randomly pivoted Cholesky chen2025randomly with rank $r = \lfloor n^{1/2} \rfloor = 141$. Partial Cholesky + Vecchia preconditioners use either $q = 0$ (PC+V0), $q = \lfloor n^{1/4} \rfloor = 11$ (PC+V1/4), or $q = \lfloor n^{1/3} \rfloor = 27$ (PC+V1/3) nonzers per row in the Vecchia component. Alternative preconditioners (Díazdiaz2024robust and Frangellafrangella2023randomized) modify randomly pivoted Cholesky by adding a multiple of the identity to the approximation and/or its nullspace.
  • Figure 1: Cholesky and inverse Cholesky decompositions of a dense matrix $\bm{A}$. Filled boxes show entries that are allowed to be nonzero.
  • Figure 1: Comparison of five pivot choosers for building partial Cholesky + diagonal (PC+V0, left) and partial Cholesky + Vecchia (PC+V1/4, right) approximations. Top row: PCG tests with kernel vectors. Middle row: PCG tests with label vectors. Bottom row: log determinant tests.
  • Figure 1: Comparison of log determinant calculations with Krylov depths $m = 1000$ (blue) and $m = 100$ (yellow).
  • Figure 2: Partial pivoted Cholesky accesses the gray-colored entries of $\bm{A}$. Here, the approximation rank is $r = 2$ and the columns $u_1 = 3$ and $u_2 = 1$ are perfectly replicated.
  • ...and 4 more figures

Theorems & Definitions (19)

  • Definition 1.1: Kaporin condition number
  • Definition 2.1: Pivoted Cholesky and inverse Cholesky decompositions
  • Definition 2.2: Partial pivoted Cholesky
  • Definition 2.3: Vecchia approximation
  • Theorem 2.4: Partial Cholesky $+$ Vecchia = Vecchia
  • Proof 1
  • Theorem 3.1: Optimality of Vecchia
  • Proposition 3.2: Approximate direct solver for linear systems
  • Proposition 3.3: Convergence of PCG axelsson2000sublinear
  • Proposition 3.4: Approximate direct solver for determinants
  • ...and 9 more