Spectral Thresholds in Correlated Spiked Models and Fundamental Limits of Partial Least Squares

TL;DR

This work develops a rigorous random matrix theory analysis of spiked cross-covariance models with partially aligned signals across two channels, formalizing the spectral behavior that underpins Partial Least Squares (PLS) in high dimensions. By analyzing the sample cross-covariance $\tilde S$ and employing resolvent and free-probability techniques, the authors establish BBP-type phase transitions for the top singular values and derive explicit thresholds via roots of cubic polynomials, together with precise asymptotics for the corresponding singular vector overlaps. They then translate these spectral results into fundamental limits for PLS, revealing a systematic gap between PLS and the Bayes-optimal estimator and identifying regimes where PLS fails to recover any signal even when detectability is possible. The findings illuminate when cross-channel correlations can aid joint recovery, compare PLS with CCA and Bayes-optimal benchmarks, and offer practical guidance for designing reliable multi-modal inference methods in high dimensions, including a rotation-based refinement to improve alignment.

Abstract

We provide a rigorous random matrix theory analysis of spiked cross-covariance models where the signals across two high-dimensional data channels are partially aligned. These models are motivated by multi-modal learning and form the standard generative setting underlying Partial Least Squares (PLS), a widely used yet theoretically underdeveloped method. We show that the leading singular values of the sample cross-covariance matrix undergo a Baik-Ben Arous-Peche (BBP)-type phase transition, and we characterize the precise thresholds for the emergence of informative components. Our results yield the first sharp asymptotic description of the signal recovery capabilities of PLS in this setting, revealing a fundamental performance gap between PLS and the Bayes-optimal estimator. In particular, we identify the SNR and correlation regimes where PLS fails to recover any signal, despite detectability being possible in principle. These findings clarify the theoretical limits of PLS and provide guidance for the design of reliable multi-modal inference methods in high dimensions.

Figures (3)

• Figure 1: Singular values of the spiked cross-covariance \ref{['eq:def_matSspk']} with one spike ($r=1$) in each channel, for high (b) and low (a) values of the signal components ($\lambda_x, \lambda_y$) while the other parameters ($\alpha_x,\alpha_y,\rho$) are the same. The figures indicate the presence of outliers outside the bulk for high values while for low values, the top two singular values stick to the edge of the distribution defined in Prop. \ref{['prop:bulk']}. The theoretical positions of the outliers for the right panel follow from our main result Thm. \ref{['thm:singularvalue']}.
• Figure 2: Empirical and theoretical values of the top two singular values of the spiked cross-covariance matrix \ref{['eq:def_matSspk']} with one spike ($r = 1$), plotted as functions of the signal strengths $\lambda_x = \lambda_y$. All other parameters ($\alpha_x$, $\alpha_y$, $\rho$) are fixed. Each empirical data point is obtained as an average over 10 sample points.
• Figure 3: Phase diagram in the $(\lambda_x,\lambda_y)$ plane illustrating the detection thresholds for Bayes-optimal method keup2024optimal (black), single-view channel SVD (gray), PLS (this paper, red), and CCA bykhovskaya2023highma2023sample (blue) algorithms. The region below each curve corresponds to values of the signal strengths for which spike detection is impossible.

Theorems & Definitions (20)

• Lemma 2.1: Roots of the cubic polynomials
• Proposition 2.1: Bulk Distribution, swain2025distributionburda2010eigenvalues
• Theorem 2.1: Phase Transition for the Singular Values
• Theorem 2.2: Phase Transition for the Singular Vectors
• Corollary 3.1
• Lemma 4.1
• Lemma 4.2
• Lemma 4.3
• Lemma 4.4
• Lemma A.1: Woodbury Resolvent identity