Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Noisy Low Rank Column-wise Sensing

Ankit Pratap Singh, Namrata Vaswani

TL;DR

This work analyzes AltGDmin for the noisy low rank column-wise sensing (LRCS) problem, deriving rigorous guarantees that extend noise-free results to the noisy setting. It provides a concrete, near-optimal sample complexity bound that improves over prior work by a factor of $\frac{\max(r, \log(1/\epsilon)) \, \log n}{r}$, and it shows that AltGDmin converges via alternating updates: a per-column least-squares update for $B$ and a gradient descent update for the factor $U$ with orthonormalization. The paper also develops a truncated spectral initialization with probabilistic guarantees and provides a comprehensive comparison to related formulations (e.g., LRPR, multi-task learning, federated sketching). Empirical results corroborate the theoretical findings, demonstrating faster convergence of AltGDmin relative to MoM initialization and random initialization in synthetic LRCS scenarios, with practical implications for federated settings and dynamic MRI.

Abstract

This letter studies the AltGDmin algorithm for solving the noisy low rank column-wise sensing (LRCS) problem. Our sample complexity guarantee improves upon the best existing one by a factor $\max(r, \log(1/ε))/r$ where $r$ is the rank of the unknown matrix and $ε$ is the final desired accuracy. A second contribution of this work is a detailed comparison of guarantees from all work that studies the exact same mathematical problem as LRCS, but refers to it by different names.

Noisy Low Rank Column-wise Sensing

TL;DR

This work analyzes AltGDmin for the noisy low rank column-wise sensing (LRCS) problem, deriving rigorous guarantees that extend noise-free results to the noisy setting. It provides a concrete, near-optimal sample complexity bound that improves over prior work by a factor of , and it shows that AltGDmin converges via alternating updates: a per-column least-squares update for and a gradient descent update for the factor with orthonormalization. The paper also develops a truncated spectral initialization with probabilistic guarantees and provides a comprehensive comparison to related formulations (e.g., LRPR, multi-task learning, federated sketching). Empirical results corroborate the theoretical findings, demonstrating faster convergence of AltGDmin relative to MoM initialization and random initialization in synthetic LRCS scenarios, with practical implications for federated settings and dynamic MRI.

Abstract

This letter studies the AltGDmin algorithm for solving the noisy low rank column-wise sensing (LRCS) problem. Our sample complexity guarantee improves upon the best existing one by a factor where is the rank of the unknown matrix and is the final desired accuracy. A second contribution of this work is a detailed comparison of guarantees from all work that studies the exact same mathematical problem as LRCS, but refers to it by different names.
Paper Structure (18 sections, 12 theorems, 55 equations, 1 figure, 1 table, 1 algorithm)

This paper contains 18 sections, 12 theorems, 55 equations, 1 figure, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Noisy LRCS Problem Setting
  3. Notation
  4. Contribution
  5. Related Work
  6. AltGDmin algorithm
  7. Discussion
  8. Experiments
  9. Proof Outline
  10. Proof outline for GD step Theorem \ref{['iters_thm']}
  11. Analyzing the minimization step for updating B
  12. Analyzing the GD step for updating U
  13. Proof of Theorem \ref{['iters_thm']}
  14. Proof outline for Initialization Theorem \ref{['init_thm']}
  15. Proof of Theorem \ref{['init_thm']}
  16. ...and 3 more sections

Key Result

Corollary 2.1

Consider Algorithm gdmin and an $0 < \epsilon < 1$. Set $\eta = c / \sigma_{1}^*{}^2$ with a $c \le 0.5$, and $T = C \kappa^2 \log(1/\epsilon)$. If and $m \ge C \max (\log q,r)\log (1/\epsilon) \max\left(1, \frac{\mathrm{NSR}}{\epsilon^2 r}\right) ,$, then, w.p. at least $1 - (T+1) n^{-10},$

Figures (1)

  • Figure 1: $\bm{SD}_2({\bm{U}}_t,{\bm{U}}^*)$ vs Iteration $t$ with $n=600$, $m=30$, $q=600$, $r=4$, and $\sigma_v^2=10^{-4}$. We compared our proposed Algorithm \ref{['gdmin']} (AltGDmin), the Method of Moments (MoM) initialization netrapalli, and a random initialization.

Theorems & Definitions (19)

  • Corollary 2.1
  • Theorem 2.2: Initialization
  • Theorem 2.3: GD
  • Lemma 3.4: Bound on $\|\bm{g}_k - \bm{b}_{k}\|$
  • Lemma 3.5: lrpr_gdmin_2
  • Lemma 3.6
  • Lemma 3.7
  • Lemma 3.8
  • proof
  • proof
  • ...and 9 more