Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Perturbation Bounds for Low-Rank Inverse Approximations under Noise

Phuc Tran, Nisheeth K. Vishnoi

TL;DR

The paper derives non-asymptotic spectral-norm bounds for the error in the best rank-$p$ inverse approximation under additive noise for symmetric matrices. Using a novel contour bootstrapping approach for the non-entire function $f(z)=1/z$, the authors bound $\| (\tilde{A}^{-1})_p - A_p^{-1} \|$ in terms of the smallest eigenvalue $\lambda_n$, the $n-p$ eigengap $\delta_{n-p}$, and the noise $E$, under the gap condition $4\|E\| \le \min\{\lambda_n, \delta_{n-p}\}$. The main result shows $\| (\tilde{A}^{-1})_p - A_p^{-1} \| \le 4\|E\|/\lambda_n^2 + 5\|E\|/(\lambda_{n-p}\delta_{n-p})$, with extensions to general symmetric matrices and refined bounds via doubling distance and interaction terms. Empirically, the proposed bounds closely track true perturbations and outperform classical Neumann/EYM-based estimates across real and synthetic matrices, offering spectrum-aware guarantees for robust low-rank inverse approximations in noisy computations.

Abstract

Low-rank pseudoinverses are widely used to approximate matrix inverses in scalable machine learning, optimization, and scientific computing. However, real-world matrices are often observed with noise, arising from sampling, sketching, and quantization. The spectral-norm robustness of low-rank inverse approximations remains poorly understood. We systematically study the spectral-norm error $\| (\tilde{A}^{-1})_p - A_p^{-1} \|$ for an $n\times n$ symmetric matrix $A$, where $A_p^{-1}$ denotes the best rank-\(p\) approximation of $A^{-1}$, and $\tilde{A} = A + E$ is a noisy observation. Under mild assumptions on the noise, we derive sharp non-asymptotic perturbation bounds that reveal how the error scales with the eigengap, spectral decay, and noise alignment with low-curvature directions of $A$. Our analysis introduces a novel application of contour integral techniques to the \emph{non-entire} function $f(z) = 1/z$, yielding bounds that improve over naive adaptations of classical full-inverse bounds by up to a factor of $\sqrt{n}$. Empirically, our bounds closely track the true perturbation error across a variety of real-world and synthetic matrices, while estimates based on classical results tend to significantly overpredict. These findings offer practical, spectrum-aware guarantees for low-rank inverse approximations in noisy computational environments.

Perturbation Bounds for Low-Rank Inverse Approximations under Noise

TL;DR

The paper derives non-asymptotic spectral-norm bounds for the error in the best rank- inverse approximation under additive noise for symmetric matrices. Using a novel contour bootstrapping approach for the non-entire function , the authors bound in terms of the smallest eigenvalue , the eigengap , and the noise , under the gap condition . The main result shows , with extensions to general symmetric matrices and refined bounds via doubling distance and interaction terms. Empirically, the proposed bounds closely track true perturbations and outperform classical Neumann/EYM-based estimates across real and synthetic matrices, offering spectrum-aware guarantees for robust low-rank inverse approximations in noisy computations.

Abstract

Low-rank pseudoinverses are widely used to approximate matrix inverses in scalable machine learning, optimization, and scientific computing. However, real-world matrices are often observed with noise, arising from sampling, sketching, and quantization. The spectral-norm robustness of low-rank inverse approximations remains poorly understood. We systematically study the spectral-norm error for an symmetric matrix , where denotes the best rank- approximation of , and is a noisy observation. Under mild assumptions on the noise, we derive sharp non-asymptotic perturbation bounds that reveal how the error scales with the eigengap, spectral decay, and noise alignment with low-curvature directions of . Our analysis introduces a novel application of contour integral techniques to the \emph{non-entire} function , yielding bounds that improve over naive adaptations of classical full-inverse bounds by up to a factor of . Empirically, our bounds closely track the true perturbation error across a variety of real-world and synthetic matrices, while estimates based on classical results tend to significantly overpredict. These findings offer practical, spectrum-aware guarantees for low-rank inverse approximations in noisy computational environments.

Paper Structure

This paper contains 31 sections, 17 theorems, 146 equations, 2 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Theoretical results
  3. Comparison to the EYM--N bound.
  4. Proof overview
  5. Extension of Theorem \ref{['theo: Apositive']} for an arbitrary symmetric matrix A
  6. Proof of Theorem \ref{['theo: ApositiveSym']}
  7. Refinements of Theorem \ref{['theo: Apositive']} and Theorem \ref{['theo: ApositiveSym']}
  8. Comparison to EYM--N bound in the intermediate regime $\min\{\lambda_n, \delta_{n-p}\} \gg \|E\| \gg \frac{\lambda_n^2}{\lambda_{n-p}}$
  9. Proof of Theorem \ref{['theo: main0']}
  10. Contour integral estimations
  11. Proof of Lemma \ref{['ingegralcomputation1']}
  12. Proof of Lemma \ref{['ingegralcomputation2']}
  13. Estimating integrals over vertical segments
  14. Estimating integrals over horizontal segments
  15. Estimating integrals over segment $L$
  16. ...and 16 more sections

Key Result

Theorem 2.1

Let $A$ be a real symmetric PD matrix and $\tilde{A} = A + E$ with $E$ symmetric. If $4\|E\| \leq \min\{\lambda_n, \delta_{n-p}\}$, then

Figures (2)

  • Figure 1: Four panels show (i) Actual Error, (ii) Our Bound, and (iii) EYM--N Bound, over 100 trials for real-world matrices $A =\text{Census}$ ($n=69$, $p=17$) and $A=\texttt{BCSSTK09}$ ($n=1083$, $p=8$), perturbed by Gaussian/Rademacher noise.
  • Figure 2: Four panels show (i) Actual Error, (ii) Our Bound, and (iii) EYM--N Bound, over 100 trials for $A=\text{Hamiltonian}$ ($p=10$, $n \in\{500,1000\}$) perturbed by Gaussian/Rademacher noise.

Theorems & Definitions (21)

  • Theorem 2.1: Main perturbation bound for PD matrices
  • Remark 2.2: A stronger but more technical bound
  • Lemma 3.1: Contour Bootstrapping
  • Theorem 4.1
  • Remark 4.2
  • Lemma 4.3
  • Lemma 4.4
  • Lemma 4.5
  • Lemma 4.6
  • Remark 4.7
  • ...and 11 more