Improved Turbo Message Passing for Compressive Robust Principal Component Analysis: Algorithm Design and Asymptotic Analysis

TL;DR

This work tackles CRPCA by casting the problem in a Bayesian framework and designing an improved turbo message passing (ITMP) algorithm that separately denoises the sparse, low-rank, and linear components. It introduces a state evolution (SE) analysis that characterizes the asymptotic MSE transfer functions for each denoiser and for the overall algorithm, enabling precise predictions of performance in the large-system limit. The authors derive both necessary and sufficient conditions for global convergence, showing that the SE-based phase-transition boundary aligns well with empirical results. Numerical experiments validate the SE predictions, confirm accurate MSE tracking across iterations, and demonstrate competitive phase-transition behavior compared to existing CRPCA methods. Overall, the paper provides a principled Bayesian ITMP design, rigorous SE analysis under mild assumptions, and practically meaningful convergence insights for CRPCA under ROIL-based compressive measurements.

Abstract

Compressive Robust Principal Component Analysis (CRPCA) naturally arises in various applications as a means to recover a low-rank matrix low-rank matrix $\boldsymbol{L}$ and a sparse matrix $\boldsymbol{S}$ from compressive measurements. In this paper, we approach the problem from a Bayesian inference perspective. We establish a probabilistic model for the problem and develop an improved turbo message passing (ITMP) algorithm based on the sum-product rule and the appropriate approximations. Additionally, we establish a state evolution framework to characterize the asymptotic behavior of the ITMP algorithm in the large-system limit. By analyzing the established state evolution, we further propose sufficient conditions for the global convergence of our algorithm. Our numerical results validate the theoretical results, demonstrating that the proposed asymptotic framework accurately characterize the dynamical behavior of the ITMP algorithm, and the phase transition curve specified by the sufficient condition agrees well with numerical simulations.

Figures (7)

• Figure 1: The factor graph representation of (\ref{['e4']})
• Figure 2: The QQplots of the input and output error in the second iteration of the proposed algorithm. The first row are the QQplots of the estimation errors input each modules. The second row are the QQplots of the estimation errors output each modules. The entries of sparse matrix $\boldsymbol{S}$ are independent drawn from the Gaussian-Bernoulli distribution with $\rho=0.1$ and the low-rank matrix $\boldsymbol{L}$ with rank-$r$. We use the point-wise MMSE denoiser to estimate $\boldsymbol{S}$ and best-rank-$r$ denoiser to estimate $\boldsymbol{L}$. We use the LMMSE denoiser to estimate $\boldsymbol{X}$. Other parameter settings are $n=n_1\times n_2=1000,\beta=1,\alpha=0.4,\sigma_{\boldsymbol{n}}^2=10^{-5},\gamma=0.05.$
• Figure 3: Illustration of the three regions in (\ref{['regdi']}) with $\alpha>\max\{\alpha_1,\alpha_2,\alpha_3\}$ and the generated $(\hat{\tau_{\boldsymbol{S}}}^{(t_{0})},\hat{\tau_{\boldsymbol{L}}}^{(t_{0})})$ from $(\tau_{\boldsymbol{S}}^{(t_{0})},\tau_{\boldsymbol{L}}^{(t_{0})})$ in the three regions $\mathcal{R}_0$, $\mathcal{R}_1$ and $\mathcal{R}_2$. The arrows represent the mapping defined in (\ref{['map_109']}). This function maps any initialization $\left(\tau_{\boldsymbol{S}}^{(t_0)}, \tau_{\boldsymbol{L}}^{(t_0)}\right)$ in $\mathcal{R}$ to a new initialization $\left(\hat{\tau}_{\boldsymbol{S}}^{(t_{0})}, \hat{\tau}_{\boldsymbol{L}}^{(t_{0})}\right)$ in $\mathcal{R}$. Here, we choose MMSE estimator for the sparse denoiser and best-rank-$r$ estimator for the low rank denoiser. The entries of $\boldsymbol{S}$ are independently generated by following a Gaussian-Bernoulli distribution with unit variance and zero mean with $\rho=0.35$. The matrix $\boldsymbol{L}$ is generated by the multiplication of two independent zero mean Gaussian matrix of sizes $n_1\times r$ and $r\times n_2$ with $\gamma=0.1$. Then, $\boldsymbol{L}$ is normalized by $\frac{1}{n}\|\boldsymbol{L}\|^2_F=1$. $\alpha=0.8$.
• Figure 4: The curve $\phi(v_{\delta\to\boldsymbol{L}})$ obtained by averaging 50 runs with a precision of $0.01$ for the variance. The other settings are $n_1=n_2=2000$.
• Figure 5: The left figure shows the dynamic changes of the estimation error of message $m_{\boldsymbol{L}\to\delta}^{(t)}$. The right figure shows the dynamic changes of the estimation error of message $m_{\boldsymbol{S}\to\delta}^{(t)}$. The settings are $n_1=n_2=1000, \gamma=0.05,\sigma_{\boldsymbol{n}}^2=0, \rho=0.05$ and different $\alpha$.

Theorems & Definitions (59)

• Definition 1: Rotationally invariant matrix
• Definition 2: ROIL
• Definition 3: Haar (distributed) matrix
• Lemma 1
• proof
• Remark 1
• Lemma 2: MMSE denoiser for $\boldsymbol{S}$
• proof
• Remark 2
• Lemma 3: Parameters for LMMSE denoiser