Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Singular Subspace Perturbation Bounds via Rectangular Random Matrix Diffusions

Peiyao Lai, Oren Mangoubi

TL;DR

This work studies how Gaussian noise affects the right-singular subspace of a rectangular matrix, focusing on the Frobenius distance between the top-$k$ subspaces of $A$ and $A+G$. The authors model the perturbation as a matrix-valued Brownian motion (the Dyson–Bessel process) and track the evolution of the right-singular vectors via stochastic differential equations, applying Ito’s lemma to obtain a bound on the perturbation of the subspace orbit $igl(V^ op Γ^2 Vigr)$. The main result provides a general bound: $Eig[ orm{reve V Γ^2 reve V^ op - V Γ^2 V^ op}_F^2ig] \\le Oigg(\sum_{i=1}^k \\sum_{j=i+1}^d rac{( ext{γ}_i^2- ext{γ}_j^2)^2}{(σ_i-σ_j)^2}igg) T$, with corollaries giving sharper subspace recovery bounds of order $ ilde{O}igl( rac{ oot{d}}{σ_k-σ_{k+1}} \\sqrt{T}igr)$ under suitable gap assumptions. This diffusion-based approach yields improved dependence on the data dimension $d$ relative to classical perturbation bounds, offering tighter utility guarantees for subspace recovery and rank-$k$ covariance approximation in Gaussian-noise regimes. The results have potential implications for statistics, signal processing, and private-data analysis where Gaussian noise is intrinsic or introduced for privacy.

Abstract

Given a matrix $A \in \mathbb{R}^{m\times d}$ with singular values $σ_1\geq \cdots \geq σ_d$, and a random matrix $G \in \mathbb{R}^{m\times d}$ with iid $N(0,T)$ entries for some $T>0$, we derive new bounds on the Frobenius distance between subspaces spanned by the top-$k$ (right) singular vectors of $A$ and $A+G$. This problem arises in numerous applications in statistics where a data matrix may be corrupted by Gaussian noise, and in the analysis of the Gaussian mechanism in differential privacy, where Gaussian noise is added to data to preserve private information. We show that, for matrices $A$ where the gaps in the top-$k$ singular values are roughly $Ω(σ_k-σ_{k+1})$ the expected Frobenius distance between the subspaces is $\tilde{O}(\frac{\sqrt{d}}{σ_k-σ_{k+1}} \times \sqrt{T})$, improving on previous bounds by a factor of $\frac{\sqrt{m}}{\sqrt{d}} \sqrt{k}$. To obtain our bounds we view the perturbation to the singular vectors as a diffusion process -- the Dyson-Bessel process -- and use tools from stochastic calculus to track the evolution of the subspace spanned by the top-$k$ singular vectors.

Singular Subspace Perturbation Bounds via Rectangular Random Matrix Diffusions

TL;DR

This work studies how Gaussian noise affects the right-singular subspace of a rectangular matrix, focusing on the Frobenius distance between the top- subspaces of and . The authors model the perturbation as a matrix-valued Brownian motion (the Dyson–Bessel process) and track the evolution of the right-singular vectors via stochastic differential equations, applying Ito’s lemma to obtain a bound on the perturbation of the subspace orbit . The main result provides a general bound: , with corollaries giving sharper subspace recovery bounds of order under suitable gap assumptions. This diffusion-based approach yields improved dependence on the data dimension relative to classical perturbation bounds, offering tighter utility guarantees for subspace recovery and rank- covariance approximation in Gaussian-noise regimes. The results have potential implications for statistics, signal processing, and private-data analysis where Gaussian noise is intrinsic or introduced for privacy.

Abstract

Given a matrix with singular values , and a random matrix with iid entries for some , we derive new bounds on the Frobenius distance between subspaces spanned by the top- (right) singular vectors of and . This problem arises in numerous applications in statistics where a data matrix may be corrupted by Gaussian noise, and in the analysis of the Gaussian mechanism in differential privacy, where Gaussian noise is added to data to preserve private information. We show that, for matrices where the gaps in the top- singular values are roughly the expected Frobenius distance between the subspaces is , improving on previous bounds by a factor of . To obtain our bounds we view the perturbation to the singular vectors as a diffusion process -- the Dyson-Bessel process -- and use tools from stochastic calculus to track the evolution of the subspace spanned by the top- singular vectors.
Paper Structure (27 sections, 11 theorems, 65 equations)

This paper contains 27 sections, 11 theorems, 65 equations.

Table of Contents

  1. Introduction
  2. Related work
  3. Our Contributions
  4. Main results
  5. Application to singular subspace recovery.
  6. Application to rank- covariance matrix approximation.
  7. Preliminaries
  8. Dyson-Bessel process
  9. Right singular vector SDE
  10. Ito's Lemma
  11. Other preliminaries
  12. Overview of proof of Theorem \ref{['ThCovariance']}
  13. Viewing the perturbed matrix as a matrix-valued Brownian motion.
  14. Projecting the matrix Brownian motion onto the orthogonal orbit $\mathcal{O}_{\Gamma^2}$.
  15. Deriving an expression for the stochastic derivative $\mathrm{d}\Psi(t)$.
  16. ...and 12 more sections

Key Result

Theorem 2.2

Let $T > 0$. Given a rectangular matrix $A\in\mathbb{R}^{m\times d}$ with singular values $\sigma_1\ge ...\ge\sigma_d\ge 0$ and corresponding orthonormal right-singular vectors $v_1,...v_d$ (and denote $V :=[v_1,...v_d]\in\mathbb{R}^{d\times d}$). Let $G$ be a matrix with i.i.d. $N(0,1)$ entries, an

Theorems & Definitions (20)

  • Theorem 2.2: Main result
  • Corollary 2.3: Subspace recovery
  • Corollary 2.4: Rank-$k$ covariance matrix approximation
  • Remark 2.5: Tightness in full-rank special case
  • Lemma 3.1: Ito's Lemma ito1951formula
  • Lemma 3.2: Weyl's Inequality weyl1912asymptotische
  • Lemma 3.3: Spectral-norm concentration bound for Gaussian matrices vershynin2018high
  • Lemma A.1: Stochastic derivative of $v_i(t)v_i(t)^\top$
  • proof
  • Lemma A.2: Stochastic derivative of $\Psi(t)$
  • ...and 10 more