Missing-Data-Induced Phase Transitions in Spectral PLS for Multimodal Learning

TL;DR

This work analyzes spectral Partial Least Squares (PLS-SVD) for multimodal learning when both views contain missing entries. By formulating a replica analysis, the authors show dual MCAR masking attenuates the cross-view spike to $\theta_{\mathrm{eff}}=\sqrt{\rho}\,\theta$, transforming the problem into a BBP-type phase transition in a spiked rectangular model with threshold $\theta_{\mathrm{crit}}=1/((\alpha_x\alpha_y)^{1/4}\sqrt{\rho})$. They derive closed-form expressions for the asymptotic overlaps above the threshold and provide a complete replica-symmetric derivation, including zero-temperature reductions to two overlaps. Extensive synthetic and semi-synthetic experiments (including TCGA BRCA and PBMC Multiome) validate the predicted phase diagram and recovery curves, demonstrating robustness to signal geometry and offering a split-half stability diagnostic for practical operation without ground-truth directions. The results deliver precise guidance on the signal strength required for recovering shared structure under missing data and motivate extensions to more general missing-data mechanisms and multi-factor two-view settings.

Abstract

Partial Least Squares (PLS) learns shared structure from paired data via the top singular vectors of the empirical cross-covariance (PLS-SVD), but multimodal datasets often have missing entries in both views. We study PLS-SVD under independent entry-wise missing-completely-at-random masking in a proportional high-dimensional spiked model. After appropriate normalization, the masked cross-covariance behaves like a spiked rectangular random matrix whose effective signal strength is attenuated by $\sqrtρ$, where $ρ$ is the joint entry retention probability. As a result, PLS-SVD exhibits a sharp BBP-type phase transition: below a critical signal-to-noise threshold the leading singular vectors are asymptotically uninformative, while above it they achieve nontrivial alignment with the latent shared directions, with closed-form asymptotic overlap formulas. Simulations and semi-synthetic multimodal experiments corroborate the predicted phase diagram and recovery curves across aspect ratios, signal strengths, and missingness levels.

Figures (7)

• Figure 1: Validation of the PLS-SVD phase transition theory under dual missingness.(a) Phase transition in squared overlaps $R_x^2$ and $R_y^2$ as signal strength $\theta$ crosses the critical threshold $\theta_{\mathrm{crit}} = 1/[(\alpha_x\alpha_y)^{1/4}\sqrt{\rho}]$ (red line). Empirical results (circles/squares with shaded error bands showing $\pm 1$ standard deviation) match theoretical predictions (dashed lines) from \ref{['thm:main']} . (b) Phase diagram in $(\theta, \rho)$ space shows empirical overlap $R_x^2$ (heatmap) with theoretical phase boundary (red dashed curve) separating the subcritical regime (recovery impossible) from the supercritical regime (successful recovery). High correlation ($r = 0.994$) confirms theoretical accuracy. (c) Finite-size sharpening: as sample size $N$ increases from 100 to 5000, empirical overlaps converge to the sharp theoretical step function (black dashed line). Parameters: (a)$N=1000$, $D_x=200$, $D_y=50$, $m_x=0.3$, $m_y=0.4$, 100 trials; (b)$N=1000$, $D_x=150$, $D_y=120$, $30\times 30$ grid, 30 trials; (c)$\alpha_x=\alpha_y=2.5$, $m_x=m_y=0.2$, 30 trials.
• Figure 2: Phase diagrams reveal distinct missingness effects on PLS-SVD recovery.(a) Single-view missingness ($m_X$ only, $m_Y=0$): the phase boundary (red dashed curve) follows $\theta_{\mathrm{crit}} = 1/[(\alpha_x\alpha_y)^{1/4}\sqrt{1-m}]$, with retention probability $\rho=1-m$ degrading linearly. (b) Joint missingness ($m_X=m_Y=m$): the boundary follows $\theta_{\mathrm{crit}} = 1/[(\alpha_x\alpha_y)^{1/4}(1-m)]$, reflecting quadratic retention degradation $\rho=(1-m)^2$. The steeper boundary demonstrates that joint missingness requires proportionally stronger signals for recovery. Heatmap shows empirical $R_x^2$ across $(\theta, m)$ parameter space. Parameters: $N=800$, $D_x=D_y=200$ ($\alpha_x=\alpha_y=4$), $50\times 50$ grid, 30 trials per point.
• Figure 3: Semi-synthetic validation with biological signal structure. Top row: TCGA BRCA (cancer genomics, $N=873$); Bottom row: PBMC Multiome (single-cell, $N=5000$). (a, d) Phase transition curves show empirical overlaps (points with shaded bands showing $\pm 1$ standard deviation) matching theoretical predictions (dashed lines). The transition occurs at $\theta/\theta_{\mathrm{crit}} = 1$ (red line). (b, e) Recovery degrades with increasing missingness, following theory (dashed lines). Fixed $\theta = 1.5\theta_{\mathrm{crit}}$. (c, f) Biological signal directions (blue) and random Gaussian directions (green) yield identical phase transitions, demonstrating universality. Theory-empirical correlation $r > 0.99$ for both datasets. All experiments use 500 trials per configuration.
• Figure 4: Split-half stability as a practical diagnostic for the phase transition. Stability (correlation between singular vectors estimated from independent data splits) distinguishes three recovery regimes. Red dashed line: full-data critical threshold $\theta_{\mathrm{crit}}$. Orange dotted line: split-half threshold $\theta_{\mathrm{crit}}^{\mathrm{half}} = \sqrt{2}\,\theta_{\mathrm{crit}}$ (each half uses $N/2$ samples, halving the aspect ratios). Shaded regions: no recovery ($\theta < \theta_{\mathrm{crit}}$, red), recovery unstable ($\theta_{\mathrm{crit}} < \theta < \sqrt{2}\,\theta_{\mathrm{crit}}$, orange), and recovery stable ($\theta > \sqrt{2}\,\theta_{\mathrm{crit}}$, green). Line color indicates true recovery quality $R_x^2$: the gradient from red (subcritical) to green (supercritical) shows that observable stability tracks the theoretically-predicted phase boundaries. This diagnostic is computable without access to ground-truth signal directions. Parameters: $N=2000$, $\alpha_x=\alpha_y=7.5$, $m_x=m_y=0.1$, 25 trials per point.
• Figure 5: Robustness of phase transition to non-Gaussian noise.(a) Phase transition in $R_x^2$ across noise types. All distributions exhibit a transition near the critical threshold $\theta_{\mathrm{crit}}$ (red line). Only the Student-$t(\nu=3)$, which has infinite kurtosis, shows noticeably reduced recovery in the supercritical regime. (b) Corresponding phase transition in $R_y^2$. (c) Theory deviation versus excess kurtosis. Deviation grows with kurtosis but remains bounded. Parameters: $N=1000$, $D_x=200$, $D_y=150$, $m_x=m_y=0.3$, 100 trials.

Theorems & Definitions (5)

• Lemma 3.1: Spiked Form under Dual Masking
• proof
• Theorem 3.2: PLS-SVD Phase Transition under Dual Masking
• Lemma A.1: Gaussian disorder average
• proof