Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spectral Perturbation Bounds for Low-Rank Approximation with Applications to Privacy

Phuc Tran, Nisheeth K. Vishnoi, Van H. Vu

TL;DR

This work develops high-probability spectral-norm perturbation bounds for the top-$p$ rank-$p$ approximation of symmetric matrices under symmetric perturbations, refining the classical Eckart–Young–Mirsky framework. The authors introduce contour bootstrapping, a complex-analytic approach that represents spectral projectors via contour integrals and yields two main bounds: a basic bound $O\left( \|E\| \frac{\lambda_p}{\delta_p} \right)$ and a refined interaction-aware bound $\tilde{O}\left( \|E\| + r^2 x \frac{\lambda_p}{\delta_p} \right)$, accounting for eigenspace alignment through $x$ and spectral decay via the halving distance $r$. The framework extends to general spectral functionals $f(A)$ and provides improved differentially private PCA guarantees with high-probability spectral-norm bounds, surpassing prior Frobenius-based results. Empirical results on real covariance matrices corroborate the sharpness of the bounds across perturbation regimes and noise models, underscoring their practical impact for privacy-preserving low-rank analysis.

Abstract

A central challenge in machine learning is to understand how noise or measurement errors affect low-rank approximations, particularly in the spectral norm. This question is especially important in differentially private low-rank approximation, where one aims to preserve the top-$p$ structure of a data-derived matrix while ensuring privacy. Prior work often analyzes Frobenius norm error or changes in reconstruction quality, but these metrics can over- or under-estimate true subspace distortion. The spectral norm, by contrast, captures worst-case directional error and provides the strongest utility guarantees. We establish new high-probability spectral-norm perturbation bounds for symmetric matrices that refine the classical Eckart--Young--Mirsky theorem and explicitly capture interactions between a matrix $A \in \mathbb{R}^{n \times n}$ and an arbitrary symmetric perturbation $E$. Under mild eigengap and norm conditions, our bounds yield sharp estimates for $\|(A + E)_p - A_p\|$, where $A_p$ is the best rank-$p$ approximation of $A$, with improvements of up to a factor of $\sqrt{n}$. As an application, we derive improved utility guarantees for differentially private PCA, resolving an open problem in the literature. Our analysis relies on a novel contour bootstrapping method from complex analysis and extends it to a broad class of spectral functionals, including polynomials and matrix exponentials. Empirical results on real-world datasets confirm that our bounds closely track the actual spectral error under diverse perturbation regimes.

Spectral Perturbation Bounds for Low-Rank Approximation with Applications to Privacy

TL;DR

This work develops high-probability spectral-norm perturbation bounds for the top- rank- approximation of symmetric matrices under symmetric perturbations, refining the classical Eckart–Young–Mirsky framework. The authors introduce contour bootstrapping, a complex-analytic approach that represents spectral projectors via contour integrals and yields two main bounds: a basic bound and a refined interaction-aware bound , accounting for eigenspace alignment through and spectral decay via the halving distance . The framework extends to general spectral functionals and provides improved differentially private PCA guarantees with high-probability spectral-norm bounds, surpassing prior Frobenius-based results. Empirical results on real covariance matrices corroborate the sharpness of the bounds across perturbation regimes and noise models, underscoring their practical impact for privacy-preserving low-rank analysis.

Abstract

A central challenge in machine learning is to understand how noise or measurement errors affect low-rank approximations, particularly in the spectral norm. This question is especially important in differentially private low-rank approximation, where one aims to preserve the top- structure of a data-derived matrix while ensuring privacy. Prior work often analyzes Frobenius norm error or changes in reconstruction quality, but these metrics can over- or under-estimate true subspace distortion. The spectral norm, by contrast, captures worst-case directional error and provides the strongest utility guarantees. We establish new high-probability spectral-norm perturbation bounds for symmetric matrices that refine the classical Eckart--Young--Mirsky theorem and explicitly capture interactions between a matrix and an arbitrary symmetric perturbation . Under mild eigengap and norm conditions, our bounds yield sharp estimates for , where is the best rank- approximation of , with improvements of up to a factor of . As an application, we derive improved utility guarantees for differentially private PCA, resolving an open problem in the literature. Our analysis relies on a novel contour bootstrapping method from complex analysis and extends it to a broad class of spectral functionals, including polynomials and matrix exponentials. Empirical results on real-world datasets confirm that our bounds closely track the actual spectral error under diverse perturbation regimes.

Paper Structure

This paper contains 27 sections, 15 theorems, 111 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Main results
  3. Proof outline
  4. Extensions of Theorem \ref{['theo: Apositive1']} and Theorem \ref{['theo: Apositive2']} to the symmetric matrices
  5. Extension of Theorem \ref{['theo: Apositive1']} to the symmetric matrices
  6. Proof of Theorem \ref{['main2']}
  7. Extension of Theorem \ref{['theo: Apositive2']} to the symmetric matrices
  8. Proof of Theorem \ref{['main2.1']}
  9. Estimating integrals over segments
  10. Proof of Lemma \ref{['ingegralcomputation1']}
  11. Estimating integrals over vertical segments for interaction-dependent bound
  12. Estimating integrals over horizontal segments
  13. Estimating integrals over vertical segments for non-interaction bound
  14. Perturbation of matrix functionals - Proof of Theorem \ref{['main3']}
  15. Empirical results
  16. ...and 12 more sections

Key Result

Theorem 2.1

If $4\|E\| \leq \delta_p$, then: $\|\tilde{A}_p - A_p\| \leq {O} ( \|E\| \cdot \frac{\lambda_p}{\delta_p})$.

Figures (3)

  • Figure 1: From Left to Right: perturbation of the $\mathrm{Census}, \mathrm{KDD}$ and $\mathrm{Adult}$ covariance matrices by Gaussian noise. Each panel plots the actual error, our bound, and the EYM bound; error bars indicate standard deviation over 100 trials.
  • Figure 2: Low-rank approximation errors under Rademacher perturbations. From left to right: the $\mathrm{Census}, \mathrm{KDD}$ and $\mathrm{Adult}$ covariance matrices.
  • Figure 3: Comparison of error metrics under Gaussian perturbation. Left: Synthetic PSD matrix with exponentially decaying spectrum ($n = 50$, $p = 5$); Center: 1990 US Census covariance matrix ($n = 69$, $p = 5$); Right: 1998 KDD-Cup covariance matrix ($n = 416$, $p = 5$). Each plot reports the spectral norm error $\|\tilde{A}_p - A_p\|$, Frobenius norm error $\|\tilde{A}_p - A_p\|_F$, and change-in-error $\left| \|A - A_p\| - \|A - \tilde{A}_p\| \right|$, as functions of Gaussian noise level $\sigma$. Error bars reflect standard deviation over 20 trials.

Theorems & Definitions (17)

  • Theorem 2.1: Main spectral bound -- PSD
  • Theorem 2.2: Interaction-aware bound
  • Theorem 2.3: Perturbation bounds for general functions
  • Corollary 2.4: Application to differential privacy
  • Lemma 3.1: Contour bootstrapping for entire function $f$
  • Remark 3.2
  • Theorem 4.1: Extension of Theorem \ref{['theo: Apositive1']} to the symmetric matrices
  • Theorem 4.1: Extension of Theorem \ref{['theo: Apositive1']} to the symmetric matrices
  • Lemma 4.2
  • Lemma 4.3
  • ...and 7 more