Discrete Double-Bracket Flows for Isotropic-Noise Invariant Eigendecomposition

TL;DR

A discrete double-bracket flow whose generator is invariant to isotropic shifts is introduced, yielding pathwise invariance to $\sigma_k^2 I$ at the discrete-time level.

Abstract

We study matrix-free eigendecomposition under a matrix-vector product (MVP) oracle, where each step observes a covariance operator $C_k = C_{sig} + σ_k^2 I + E_k$. Standard stochastic approximation methods either use fixed steps that couple stability to $\|C_k\|_2$, or adapt steps in ways that slow down due to vanishing updates. We introduce a discrete double-bracket flow whose generator is invariant to isotropic shifts, yielding pathwise invariance to $σ_k^2 I$ at the discrete-time level. The resulting trajectory and a maximal stable step size $η_{max} \propto 1/\|C_e\|_2^2$ depend only on the trace-free covariance $C_e$. We establish global convergence via strict-saddle geometry for the diagonalization objective and an input-to-state stability analysis, with sample complexity scaling as $O(\|C_e\|_2^2 / (Δ^2 ε))$ under trace-free perturbations. An explicit characterization of degenerate blocks yields an accelerated $O(\log(1/ζ))$ saddle-escape rate and a high-probability finite-time convergence guarantee.

Figures (9)

• Figure 1: Iterations to convergence vs. $\sigma^2$ ($n=10$, 5 seeds). Left: All methods. Commutator methods remain flat at $\approx 1600$ iterations; Raw Oja fails for $\sigma^2 \ge 10$. Right: Commutator only, confirming $O(1)$ complexity.
• Figure 2: Mechanism diagnostics. (a) Oja and commutator directions are uncorrelated: $P(\cos\theta < 0) = 0.50$. (b) SI contraction ratio $\rho \to 1$ as $\sigma^2 \to \infty$. (c) Haar-distributed $M_0$ yields $\mathbb{E}[\delta(M_0)] \approx 0.016g$. (d) Wall-clock crossover at $\sigma^2 \approx 300$.
• Figure 3: E1: Convergence trajectories for $\sigma^2 \in \{0, 10^3, 10^6\}$. All three curves overlap exactly (within machine precision), demonstrating pathwise $\sigma^2$-invariance.
• Figure 4: E14: Iterations to convergence vs $\sigma^2$. Commutator methods achieve $O(1)$ complexity (flat line); Raw Oja exhibits $O(\sigma^2)$ degradation and fails for $\sigma^2 \ge 10$.
• Figure 5: E3: ISS steady-state error scales linearly with noise amplitude $\varepsilon_E$, independent of $\sigma^2$. All three $\sigma^2$ curves overlap with $R^2 > 0.93$.

Theorems & Definitions (235)

• Theorem 3.1: $\sigma^2$-Immunity
• Lemma 4.1: Spectral Sandwiching
• Lemma 4.2: Discrete Lyapunov Descent
• Theorem 4.3: Discrete ISS
• Lemma 4.4: Domain Radius
• Theorem 4.5: Non-Escape Condition
• Theorem 4.6: Sample Complexity
• Lemma 5.1: Critical Point Characterization
• Lemma 5.2: Critical Point Dichotomy
• proof