Orthogonal Approximate Message Passing with Optimal Spectral Initializations for Rectangular Spiked Matrix Models

TL;DR

The paper develops a rigorous orthogonal AMP framework for rectangular spiked matrix models with general rotationally invariant noise, establishing a state evolution that enables Bayes-optimal scalar and matrix denoisers. It introduces optimal spectral initialization that aggregates multiple informative outliers and presents data-driven methods to realize oracle performance, including mechanisms to resolve relative signs across outliers. By integrating spectral initializers into OAMP, the authors provide a principled SE for spectrally initialized OAMP and demonstrate consistency with replica-symmetric Bayes predictions in regimes without computational-gap limitations. The results offer a robust approach for high-dimensional signal estimation under non-Gaussian RI noise, with substantial implications for principal component analysis in multi-outlier settings and for spectral methods in RI ensembles.

Abstract

We propose an orthogonal approximate message passing (OAMP) algorithm for signal estimation in the rectangular spiked matrix model with general rotationally invariant (RI) noise. We establish a rigorous state evolution that precisely characterizes the algorithm's high-dimensional dynamics and enables the construction of iteration-wise optimal denoisers. Within this framework, we accommodate spectral initializations under minimal assumptions on the empirical noise spectrum. In the rectangular setting, where a single rank-one component typically generates multiple informative outliers, we further propose a procedure for combining these outliers under mild non-Gaussian signal assumptions. For general RI noise models, the predicted performance of the proposed optimal OAMP algorithm agrees with replica-symmetric predictions for the associated Bayes-optimal estimator, and we conjecture that it is statistically optimal within a broad class of iterative estimation methods.

Figures (6)

• Figure 1: Spectral behavior of the rectangular spiked model in super critical $\theta$-regime under different noise distributions. Left: Gaussian noise; the bulk follows the Marčenko--Pastur density $\mu_1(\lambda)$ and exhibits a single outlier. Center: Non-Gaussian noise with bulk $\mu_2(\lambda)=\frac{2}{\pi}\sqrt{(\lambda-2)(4-\lambda)}\,\mathbf{1}_{[2,4]}(\lambda)$, producing two outliers. Right: Non-Gaussian noise with bulk $\mu_3(\lambda)=\frac{1}{\pi}\sqrt{(\lambda-2)(4-\lambda)}\,\mathbf{1}_{[2,4]}(\lambda)+\frac{1}{\pi}\sqrt{(\lambda-6)(8-\lambda)}\,\mathbf{1}_{[6,8]}(\lambda)$, producing multiple outliers. Dashed vertical lines indicate the real roots of the master equation in Lemma \ref{['lem:Gamma-analytic']}.
• Figure 2: Relative-sign estimation via MLE and NGMC. (a)--(b): MLE with Rademacher $\mathsf U_*$ and Gaussian $\mathsf V_*$ under noise law $\mu_2$ ($\delta=0.7$). (c)--(d): NGMC with Student-$t$$\mathsf U_*$ (df$=5$) under noise law $\mu_3$ ($\delta=0.8$). Vertical green lines indicate the SNR phase transitions where successive outliers detach from the bulk (cf. Fig. \ref{['fig:combined-outliers']}). Across all experiments, $N=5000$ and results are averaged over $50$ trials. Stars denote the proposed estimators; circles denote PCA; the solid line denotes the oracle bound.
• Figure 3: Performance under i.i.d. Gaussian noise ($N=8000,\delta=0.6$, $\theta=2$). Markers are empirical averages over $50$ trials. All methods are initialized with cosine similarity $0.2$ in both channels.
• Figure 4: Signed cosine similarity of spectrally-initialized OAMP under RI noise with bulk density $\mu_2(\lambda)=\frac{2}{\pi}\sqrt{(\lambda-2)(4-\lambda)}\,\mathbf{1}_{[2,4]}(\lambda)$ (cf. Fig. \ref{['fig:combined-outliers']}), with $\delta=0.7$ and $\theta=1$. Left: asymmetric three-point prior. Right: symmetric Rademacher prior. In both cases $N=20000$. Stars and circles represent the two global orientations of the spectral initializer (initial overlap positive vs. negative).
• Figure 5: Non-Gaussian RI noise with bulk density $\mu_4(\lambda)=\frac{2}{\pi}\sqrt{(\lambda-1)(3-\lambda)}\,\mathbf{1}_{[1,3]}(\lambda)$ and Rademacher priors ($N=10000$, $\delta=0.5$). Markers are empirical averages over $50$ trials. When outlier eigenvectors are present, OAMP denotes the spectrally-initialized optimal OAMP in Theorem \ref{['thm: spec-SE']}, whereas AMP uses the spectral initialization of zhong2021approximate based only on the top eigenvector. When no outlier eigenvectors are available, all methods are initialized with cosine similarity $0.1$.

Theorems & Definitions (87)

• Definition 1: Wasserstein convergence
• Definition 2: Asymptotic equivalence
• Definition 3: Stieltjes transform of a finite signed measure
• Definition 4: C-transform
• Definition 5: Signal–eigenspace spectral measures
• Lemma 1: Analytic structure and zeros of the master equation
• proof
• Lemma 2
• proof
• Proposition 1: Outlier characterization