Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Precision-induced Adaptive Randomized Low-Rank Approximation for SVD and Matrix Inversion

Weiwei Xu, Weijie Shen, Zhengjian Bai, Chen Xu

TL;DR

This work tackles the high cost of SVD and matrix inversion on large, potentially ill-conditioned matrices by introducing a precision-driven framework governed by the $\\oldsymbol{\\epsilon}$-rank. A Gaussian random re-normalization approach adaptively determines the effective rank without manual tuning, enabling SVD and inversion with complexity near $O(mn r_{\\epsilon})$ and error scales on the order of $O(\\sqrt{\\epsilon})$. The authors develop GRSVD and GRI (with variants GBRI/GBRI) that project computations to a small subspace, backed by rigorous probabilistic guarantees and Sherman–Morrison–Woodbury-based inversion formulas. Numerical experiments demonstrate substantial speedups (often an order of magnitude) while preserving reconstruction quality, making the methods practical for large-scale, nearly low-rank problems common in science and engineering.

Abstract

Singular value decomposition (SVD) and matrix inversion are ubiquitous in scientific computing. Both tasks are computationally demanding for large scale matrices. Existing algorithms can approximatively solve these problems with a given rank, which however is unknown in practice and requires considerable cost for tuning. In this paper, we tackle the SVD and matrix inversion problems from a new angle, where the optimal rank for the approximate solution is explicitly guided by the distribution of the singular values. Under the framework, we propose a precision-induced random re-normalization procedure for the considered problems without the need of guessing a good rank. The new algorithms built upon the procedure simultaneously calculate the optimal rank for the task at a desired precision level and lead to the corresponding approximate solution with a substantially reduced computational cost. The promising performance of the new algorithms is supported by both theory and numerical examples.

Precision-induced Adaptive Randomized Low-Rank Approximation for SVD and Matrix Inversion

TL;DR

This work tackles the high cost of SVD and matrix inversion on large, potentially ill-conditioned matrices by introducing a precision-driven framework governed by the -rank. A Gaussian random re-normalization approach adaptively determines the effective rank without manual tuning, enabling SVD and inversion with complexity near and error scales on the order of . The authors develop GRSVD and GRI (with variants GBRI/GBRI) that project computations to a small subspace, backed by rigorous probabilistic guarantees and Sherman–Morrison–Woodbury-based inversion formulas. Numerical experiments demonstrate substantial speedups (often an order of magnitude) while preserving reconstruction quality, making the methods practical for large-scale, nearly low-rank problems common in science and engineering.

Abstract

Singular value decomposition (SVD) and matrix inversion are ubiquitous in scientific computing. Both tasks are computationally demanding for large scale matrices. Existing algorithms can approximatively solve these problems with a given rank, which however is unknown in practice and requires considerable cost for tuning. In this paper, we tackle the SVD and matrix inversion problems from a new angle, where the optimal rank for the approximate solution is explicitly guided by the distribution of the singular values. Under the framework, we propose a precision-induced random re-normalization procedure for the considered problems without the need of guessing a good rank. The new algorithms built upon the procedure simultaneously calculate the optimal rank for the task at a desired precision level and lead to the corresponding approximate solution with a substantially reduced computational cost. The promising performance of the new algorithms is supported by both theory and numerical examples.
Paper Structure (14 sections, 12 theorems, 56 equations, 4 figures, 4 tables, 6 algorithms)

This paper contains 14 sections, 12 theorems, 56 equations, 4 figures, 4 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Notations
  3. Random re-normalization
  4. Gaussian random re-normalization
  5. Adaptive determination of $\epsilon$-rank
  6. Accelerating SVD of nearly low-rank matrix
  7. Accelerating matrix inversion of two matrix inverses
  8. Numerical examples
  9. Concluding remarks
  10. Invariance of rank under multiplication by full-rank matrices
  11. Expectation of Norms
  12. Proof of Lemma \ref{['Thm.Expect']}
  13. Proof of Lemma \ref{['vq0']}
  14. Proof of Theorem \ref{['l2']}

Key Result

Lemma 2.2

\newlabelThm.Gaussian Given a matrix $A\in\mathbb{C}^{m\times n}$, any Gaussian random matrix $\Omega\in\mathbb{C}^{n\times k}$ for all $\mathrm{rank}(A) \leq k\leq n$ is random re-normalization matrix of matrix $A$.

Figures (4)

  • Figure 5.1: Calculation time of SVD for nearly low-rank matrices.
  • Figure 5.2: Calculation time of matrix inverse.
  • Figure 5.3: Schematic of $\epsilon$-rank denoising using eSVD, randomized SVD, GRSVD, Algorithm 4.2 in 9 and randQB-EI. (a) A 3-D stack of echocardiography. (b) A spatiotemporal representation (Casorati matrix) where all pixels at one time point are arranged in one column. (c) Singular value curves of eSVD, RSVD and GRSVD. (d) Stem image of the cumulative energy ratio of eSVD. (e) Stem image of the cumulative energy ratio of RSVD. (f) Stem image of the cumulative energy ratio of GRSVD.
  • Figure 5.4: Echocardiographic images processed by different methods. (a) Original. (b) GRSVD. (c) eSVD. (d) RSVD. (e) ARRF. (f) randQB-EI.

Theorems & Definitions (25)

  • Definition 1.1: $\epsilon$-rank
  • Definition 2.1: Random re-normalization matrix
  • Remark 2.1
  • Lemma 2.2
  • proof
  • Lemma 2.3
  • Lemma 2.4
  • Remark 2.2
  • Theorem 3.1
  • proof
  • ...and 15 more