Randomized low-rank approximation of parameter-dependent matrices

TL;DR

The paper develops a framework for randomized low-rank approximation of parameter-dependent matrices by using constant dimension reduction matrices, enabling efficient offline/online computation for families of matrices $A(t)$. By extending the Halko–Martinsson–Tropp (HMT) method and the generalized Nyström method to parameterized settings and leveraging affine decompositions $A(t)=\sum_i \varphi_i(t) A_i$, it derives probabilistic error bounds in $L^2$ and in uniform norms, as well as corresponding tail probabilities. Numerical experiments across synthetic problems, a parametric cookie PDE, a discrete Schrödinger system, and a parameterized Gaussian covariance kernel demonstrate that constant DRMs incur only modest loss in accuracy relative to independent DRMs, while offering favorable computational cost. The work thus provides a practical, theoretically-grounded approach for scalable, smooth, and accurate low-rank approximations of large, parameter-dependent matrices with potential impact in statistics, dynamical systems, and kernel methods.

Abstract

This work considers the low-rank approximation of a matrix $A(t)$ depending on a parameter $t$ in a compact set $D \subset \mathbb{R}^d$. Application areas that give rise to such problems include computational statistics and dynamical systems. Randomized algorithms are an increasingly popular approach for performing low-rank approximation and they usually proceed by multiplying the matrix with random dimension reduction matrices (DRMs). Applying such algorithms directly to $A(t)$ would involve different, independent DRMs for every $t$, which is not only expensive but also leads to inherently non-smooth approximations. In this work, we propose to use constant DRMs, that is, $A(t)$ is multiplied with the same DRM for every $t$. The resulting parameter-dependent extensions of two popular randomized algorithms, the randomized singular value decomposition and the generalized Nyström method, are computationally attractive, especially when $A(t)$ admits an affine linear decomposition with respect to $t$. We perform a probabilistic analysis for both algorithms, deriving bounds on the expected value as well as failure probabilities for the approximation error when using Gaussian random DRMs. Both, the theoretical results and numerical experiments, show that the use of constant DRMs does not impair their effectiveness; our methods reliably return quasi-best low-rank approximations.

Figures (11)

• Figure 1: Synthetic example \ref{['eq:syntheticexample']} with $n = 100$. $L^2$ approximation error of parameter-dependent HMT / Nyström compared to the pointwise truncated SVD for different approximation ranks.
• Figure 2: Synthetic example \ref{['eq:syntheticexample']} with $n = 100$. $L^2$ approximation error of parameter-dependent HMT / Nyström (green) compared to pointwise HMT / Nyström with different, independent DRMs (blue) for approximation ranks $10$, $20$, $30$, and $40$.
• Figure 3: Singular values of the reference solution to \ref{['eq:paracookie']} at $t=0,0.45$ and $0.9$.
• Figure 4: Parametric cookie example \ref{['eq:paracookie']}. $L^2$ approximation error of parameter-dependent HMT / Nyström compared to the pointwise truncated SVD for different approximation ranks.
• Figure 5: The singular values of the reference solution to the discrete Schrödinger equation at $t=0,0.05$ and $0.1$.

Theorems & Definitions (29)

• Theorem 1: halko2011finding
• Theorem 2: halko2011finding
• Theorem 3: halko2011finding
• Lemma 4
• proof
• Remark 1
• Theorem 5
• proof
• Lemma 6
• proof