Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Sherman--Morrison--Woodbury approach to solving least squares problems with low-rank updates

Stefan Güttel, Yuji Nakatsukasa, Marcus Webb, Alban Bloor Riley

TL;DR

This work generalizes the Sherman--Morrison--Woodbury framework to update the Moore--Penrose pseudoinverse under a low-rank change $A+UV^T$, producing a practical algorithm (WoodburyLS) for updated least-squares problems. The key contribution is the explicit update formula $(A+UV^T)^ abla = (I-M) A^ abla + (I-M)(A^TA)^{-1} VU^T$, with $M=(A^TA)^{-1}X(I+Y^T(A^TA)^{-1}X)^{-1}Y^T$, $X=[V, A^T U]$, and $Y=[(A+UV^T)^T U, V]$, which is generically a rank-$2r$ correction. WoodburyLS solves the updated problem using only $2r$ solves for $(A^TA)^{-1}X$ and $2r$ solves for $A^TA$, plus the base $A$-solve, enabling substantial speedups when a QR factorization of $A$ is available (roughly $O(n/r)$ over the naive approach). Numerical tests show 20x–130x speedups with backward errors near machine precision, validating the method’s practical impact for deflation in Gauss--Newton-like LS steps and other repeated low-rank updates.

Abstract

We present a simple formula to update the pseudoinverse of a full-rank rectangular matrix that undergoes a low-rank modification, and demonstrate its utility for solving least squares problems. The resulting algorithm can be dramatically faster than solving the modified least squares problem from scratch, just like the speedup enabled by Sherman--Morrison--Woodbury for solving linear systems with low-rank modifications.

A Sherman--Morrison--Woodbury approach to solving least squares problems with low-rank updates

TL;DR

This work generalizes the Sherman--Morrison--Woodbury framework to update the Moore--Penrose pseudoinverse under a low-rank change , producing a practical algorithm (WoodburyLS) for updated least-squares problems. The key contribution is the explicit update formula , with , , and , which is generically a rank- correction. WoodburyLS solves the updated problem using only solves for and solves for , plus the base -solve, enabling substantial speedups when a QR factorization of is available (roughly over the naive approach). Numerical tests show 20x–130x speedups with backward errors near machine precision, validating the method’s practical impact for deflation in Gauss--Newton-like LS steps and other repeated low-rank updates.

Abstract

We present a simple formula to update the pseudoinverse of a full-rank rectangular matrix that undergoes a low-rank modification, and demonstrate its utility for solving least squares problems. The resulting algorithm can be dramatically faster than solving the modified least squares problem from scratch, just like the speedup enabled by Sherman--Morrison--Woodbury for solving linear systems with low-rank modifications.
Paper Structure (5 sections, 1 theorem, 11 equations, 3 figures, 1 algorithm)

This paper contains 5 sections, 1 theorem, 11 equations, 3 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. An update formula for the pseudoinverse
  3. Solving updated least squares problems
  4. Solving for multiple low-rank updates and right-hand sides
  5. Numerical experiment

Key Result

Theorem 1

\newlabelthm:BGNW0 Let $A\in\mathbb{R}^{m\times n}$ and $U\in\mathbb{R}^{m\times r}, V\in\mathbb{R}^{n\times r}$ with $m\geq n \geq r$. Assume that $\mathop{\mathrm{rank}}\nolimits(A)=\mathop{\mathrm{rank}}\nolimits(A+UV^T) = n$. Then where

Figures (3)

  • Figure 1: MATLAB function implementing WoodburyLS described in Algorithm \ref{['alg:WoodburyLS']}. \newlabelmatlabcode0
  • Figure 1: \newlabelfig:code0 An example demonstrating the use of WoodburyLS in MATLAB. Both x1 and x2 are solutions to the modified least squares problem $\min_x \| b - (A+UV^T)x \|_2$, but x2 is computed significantly faster.
  • Figure 2: Speedup obtained with WoodburyLS compared to solving the modified least squares problem from scratch by computing a QR factorization of $A+U V^T$.

Theorems & Definitions (2)

  • Theorem 1
  • Proof 1