Orthogonal Approximate Message Passing Algorithms for Rectangular Spiked Matrix Models with Rotationally Invariant Noise

TL;DR

This work extends orthogonal AMP to rectangular spiked matrix models with rotationally invariant noise by deriving a rigorous state evolution that exactly tracks high-dimensional dynamics. It introduces an optimal OAMP variant with MMSE-based denoisers and a spectral pre-processing framework that leverages the noise spectrum, showing that in the Gaussian RI case the fixed point aligns with AMP and conjecturing Bayes-optimal performance in broader RI settings. Theoretical SE results are complemented by simulations under IID Gaussian and non-Gaussian RI noise, which validate the predictions and demonstrate superior performance over PCA and RI-AMP baselines. Overall, the paper provides a principled, spectrally informed approach to achieving near-optimal signal recovery in high-dimensional, RI-noise environments with rectangular data geometry.

Abstract

We propose an orthogonal approximate message passing (OAMP) algorithm for signal estimation in the rectangular spiked matrix model with general rotationally invariant (RI) noise. We establish a rigorous state evolution that exactly characterizes the high-dimensional dynamics of the algorithm. Building on this framework, we derive an optimal variant of OAMP that minimizes the predicted mean-squared error at each iteration. For the special case of i.i.d. Gaussian noise, the fixed point of the proposed OAMP algorithm coincides with that of the standard AMP algorithm. For general RI noise models, we conjecture that the optimal OAMP algorithm is statistically optimal within a broad class of iterative methods, and achieves Bayes-optimal performance in certain regimes.

Figures (2)

• Figure 1: Simulation results for the i.i.d. Gaussian noise model. The empirical results (markers) are averaged over $50$ runs with $M=4000, N=8000$ and $\theta=2$. The algorithm is initialized with a prior cosine similarity of $0.2$ for both signals.
• Figure 2: Simulation results for the non-Gaussian noise case. The eigenvalues of $\bm{WW}^\top$ are drawn from a rescaled/shifted Beta(1.5, 1.5) distribution with support on $[1, 3]$. The signal components are from a Rademacher prior. Other settings are the same as in Fig. \ref{['fig:res']}.

Theorems & Definitions (6)

• Definition 1: Signal–Eigenspace Spectral Measures
• Lemma 1
• Definition 2: OAMP Algorithms
• Theorem 1: State Evolution
• Proposition 1
• Proposition 2