Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Matrices with Gaussian noise: optimal estimates for singular subspace perturbation

Sean O'Rourke, Van Vu, Ke Wang

Abstract

The Davis-Kahan-Wedin $\sin Θ$ theorem describes how the singular subspaces of a matrix change when subjected to a small perturbation. This classic result is sharp in the worst case scenario. In this paper, we prove a stochastic version of the Davis-Kahan-Wedin $\sin Θ$ theorem when the perturbation is a Gaussian random matrix. Under certain structural assumptions, we obtain an optimal bound that significantly improves upon the classic Davis-Kahan-Wedin $\sin Θ$ theorem. One of our key tools is a new perturbation bound for the singular values, which may be of independent interest.

Matrices with Gaussian noise: optimal estimates for singular subspace perturbation

Abstract

The Davis-Kahan-Wedin theorem describes how the singular subspaces of a matrix change when subjected to a small perturbation. This classic result is sharp in the worst case scenario. In this paper, we prove a stochastic version of the Davis-Kahan-Wedin theorem when the perturbation is a Gaussian random matrix. Under certain structural assumptions, we obtain an optimal bound that significantly improves upon the classic Davis-Kahan-Wedin theorem. One of our key tools is a new perturbation bound for the singular values, which may be of independent interest.

Paper Structure

This paper contains 26 sections, 26 theorems, 279 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Main results
  3. Individual singular vector bounds
  4. Singular subspace bounds
  5. Comparison to other results in the literature
  6. Outline and notation
  7. Basic tools and an overview of the proof
  8. Linear algebra
  9. Resolvent
  10. Singular value locations
  11. Additional tools
  12. Overview of the proofs
  13. Proof of Theorem \ref{['thm:subspace']}
  14. Proof of Theorem \ref{['thm:singularlocation']}
  15. Proof of Lemma \ref{['lemma:assumpGOE']}
  16. ...and 11 more sections

Key Result

Theorem 1

One has where $\angle (u_1, \widetilde{u}_1)$ is the acute angle between $u_1$ and $\widetilde{u}_1$, taken in $[0, \pi/2]$. The same bound holds for $\sin \angle (v_1, \widetilde{v}_1)$.

Figures (2)

  • Figure 1: A plot of the cumulative distribution function $F$ of $\sin\angle(u_1,\widetilde{u}_1)$, where $F(x) = \mathbb{P}(\sin\angle(u_1,\widetilde{u}_1) \leq x)$ for $0 \leq x \leq 1$. We take $N=n=1000$ and $A=\mathop{\mathrm{diag}}\nolimits(300,300-\delta,0,\cdots,0)$ with rank $r=2$ where the spectral gap $\delta=\delta_1$ is chosen to be $20$, $10$, $5$, and $2$. $E$ is a Gaussian matrix as in Theorem \ref{['thm:gaussian-friendly']}. Each curve is generated from $400$ samples.
  • Figure 2: Distinct circles $\mathcal{C}_j$ with centers $z_j$ on the real line and radius $20\eta r$ for $i_0 \le j \le r_0$.

Theorems & Definitions (38)

  • Theorem 1: Davis--Kahan--Wedin $\sin \Theta$ theorem
  • Theorem 2: Perturbation with Gaussian noise; simplified asymptotic version
  • Theorem 3: Lower bound
  • Theorem 4: Perturbation with Gaussian noise
  • Remark 5
  • Theorem 6: Singular subspace bounds; simplified asymptotic version
  • Theorem 7: Singular subspace bounds
  • Proposition 8
  • Lemma 9
  • Proposition 10
  • ...and 28 more