Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Approximating Sparse Matrices and their Functions using Matrix-vector products

Taejun Park, Yuji Nakatsukasa

TL;DR

It is shown that when $A$ is a sparse matrix with an unknown sparsity pattern, techniques from compressed sensing can be used under natural assumptions and certain deterministic matrix-vector products can efficiently recover the large entries of $f(A)$.

Abstract

The computation of a matrix function $f(A)$ is an important task in scientific computing appearing in machine learning, network analysis and the solution of partial differential equations. In this work, we use only matrix-vector products $x\mapsto Ax$ to approximate functions of sparse matrices and matrices with similar structures such as sparse matrices $A$ themselves or matrices that have a similar decay property as matrix functions. We show that when $A$ is a sparse matrix with an unknown sparsity pattern, techniques from compressed sensing can be used under natural assumptions. Moreover, if $A$ is a banded matrix then certain deterministic matrix-vector products can efficiently recover the large entries of $f(A)$. We describe an algorithm for each of the two cases and give error analysis based on the decay bound for the entries of $f(A)$. We finish with numerical experiments showing the accuracy of our algorithms.

Approximating Sparse Matrices and their Functions using Matrix-vector products

TL;DR

It is shown that when is a sparse matrix with an unknown sparsity pattern, techniques from compressed sensing can be used under natural assumptions and certain deterministic matrix-vector products can efficiently recover the large entries of .

Abstract

The computation of a matrix function is an important task in scientific computing appearing in machine learning, network analysis and the solution of partial differential equations. In this work, we use only matrix-vector products to approximate functions of sparse matrices and matrices with similar structures such as sparse matrices themselves or matrices that have a similar decay property as matrix functions. We show that when is a sparse matrix with an unknown sparsity pattern, techniques from compressed sensing can be used under natural assumptions. Moreover, if is a banded matrix then certain deterministic matrix-vector products can efficiently recover the large entries of . We describe an algorithm for each of the two cases and give error analysis based on the decay bound for the entries of . We finish with numerical experiments showing the accuracy of our algorithms.
Paper Structure (27 sections, 3 theorems, 50 equations, 6 figures, 2 algorithms)

This paper contains 27 sections, 3 theorems, 50 equations, 6 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Existing methods
  3. Contributions
  4. Notation
  5. Approximation for functions of general sparse matrices
  6. Strategy for recovering general sparse matrices
  7. Complexity of SpaMRAM (Alg. \ref{['alg:gen']})
  8. Dependency of the compressed sensing solver in SpaMRAM
  9. A posteriori error estimate for SpaMRAM
  10. Adaptive methods
  11. Shortcomings
  12. Analysis of SpaMRAM (Alg. \ref{['alg:gen']})
  13. Approximation for functions of banded matrices
  14. Strategy for recovering banded matrices
  15. Example
  16. ...and 12 more sections

Key Result

Theorem 1

Let $\bm{B}\in\mathbb{R}^{n\times n}$ be a matrix satisfying for constants $C_B>0$, $1>\lambda_B>0$ and $d(i,j) \in \mathbb{R}_{\geq 0} \cup \{+\infty\}$ is some bivariate function that captures the magnitude of the entries of $\bm{B}$. Then the output of SpaMRAM, $\widehat{\bm{B}}$, satisfies where is the sparsity parameter in Algorithm alg:gen with the distance parameter $d$ and the compresse

Figures (6)

  • Figure 1: $f$ is the exponential function $e^x$ and $\bm{A}\in \mathbb{R}^{1000\times 1000}$ is a symmetric $2$-banded matrix. The left figure shows the exponential decay in the entries of $f(\bm{A})$ and the right figure shows that the error analysis in \ref{['erranal']} captures the decay rate until the error is dominated by the error from evaluating matrix-vector products using the Krylov method. The stagnation of the $\max\limits_{i = 1,2,...,n} \left|[f(\bm{A})]_{i,i\pm (s-1)/2}\right|$ curve is caused by machine precision. The exponential decay in the max-norm error will be used to derive the $2$-norm error bound in Section \ref{['subsec:bandanal']}.
  • Figure 2: The accuracy of BaMRAM and SpaMRAM using two $1024\times 1024$ synthetic symmetric matrices: a random $2$-banded matrix (left) and a random sparse matrix with nonzero density $1/1024$ (right). The dotted lines are the error estimates $\delta_{RE}^{(S)}$ and $\delta_{RE}^{(B)}$ discussed in Sections \ref{['subsec:posterrspamram']} and \ref{['subsec:errestband']} respectively. The error estimates can be used to determine the behaviour of SpaMRAM and BaMRAM.
  • Figure 3: The accuracy of BaMRAM and SpaMRAM for functions of matrices when $f(\bm{A})$ itself is banded or sparse.
  • Figure 4: Eigenvalue spectrum of $\mathtt{west1505}$ and $\mathtt{tols2000}$
  • Figure 5: The accuracy of BaMRAM (left) and SpaMRAM (right) using sparse matrices from FloridaDataset2011.
  • ...and 1 more figures

Theorems & Definitions (8)

  • Theorem 1: Algorithm \ref{['alg:gen']}
  • proof
  • Remark 2
  • Proposition 3
  • proof
  • Theorem 4: Algorithm \ref{['alg:band']}
  • proof
  • Remark 5