Hybrid Partial Least Squares Regression with Multiple Functional and Scalar Predictors

TL;DR

This work introduces Hybrid PLS, a framework that jointly models multiple functional and scalar predictors within a unified hybrid Hilbert space, enabling supervised dimensionality reduction and regression under intermodality correlations. It extends the nonlinear iterative PLS (NIPALS) algorithm to the hybrid setting, deriving a population eigenproblem and a sample-level procedure with a closed-form for the regularized PLS directions and a residualization-based update, while enforcing orthogonality of scores and directions via a roughness-penalized inner product. The authors establish the theoretical well-posedness and convergence of the estimator, and validate the approach through extensive simulations and an application to renal renogram data, demonstrating improved predictive power and parsimonious low-dimensional representations in the presence of multicollinearity. The practical impact lies in robustly leveraging cross-modality information (functional curves and scalar covariates) for prediction and inference in biomedical settings, with a computationally efficient algorithm that handles dense or irregular functional data.

Abstract

Motivated by renal imaging studies that combine renogram curves with pharmacokinetic and demographic covariates, we propose Hybrid partial least squares (Hybrid PLS) for simultaneous supervised dimension reduction and regression in the presence of cross-modality correlations. The proposed approach embeds multiple functional and scalar predictors into a unified hybrid Hilbert space and rigorously extends the nonlinear iterative PLS (NIPALS) algorithm. This theoretical development is complemented by a sample-level algorithm that incorporates roughness penalties to control smoothness. By exploiting the rank-one structure of the resulting optimization problem, the algorithm admits a computationally efficient closed-form solution that requires solving only linear systems at each iteration. We establish fundamental geometric properties of the proposed framework, including orthogonality of the latent scores and PLS directions. Extensive numerical studies on synthetic data, together with an application to a renal imaging study, validate these theoretical results and demonstrate the method's ability to recover predictive structure under intermodal multicollinearity, yielding parsimonious low-dimensional representations.

Figures (8)

• Figure 1: Typical renogram profiles from the Emory University renal study. For each kidney, two functional curves are recorded: (a) the baseline renogram curve and (b) the post-furosemide renogram curve. Lines represent three example cases: nonobstructed (solid), obstructed (dashed), and equivocal (dotted). (c) Heatmap of the correlation matrix for all predictors across 216 subjects, highlighting the strong dependence between the two functional modalities and the fourteen scalar covariates.
• Figure 2: Mean absolute sample correlations between the response vector $Y_1, \ldots, Y_n$ and the latent PLS scores $\widehat{\rho}_1^{[l]}, \ldots, \widehat{\rho}_n^{[l]}$ for $l = 1, \ldots, 5$, averaged over 100 Monte Carlo replications. Error bars indicate one standard deviation.
• Figure 3: Sensitivity of relative estimation errors to the roughness penalties $\lambda_1$ and $\lambda_2$. Top and middle rows show the relative $\mathbb{L}^2$ error for functional coefficients $\beta_1(t)$ and $\beta_2(t)$, while the bottom row shows the relative $\ell_2$ error for the scalar coefficient $\boldsymbol{\beta}$. Columns correspond to sample sizes $n = 200, 1000, 3000$. The $x$-axis represents $\lambda_1$, with line types indicating $\lambda_2 \in \{0, 0.001, 0.1\}$. Reported values are mean across 100 Monte Carlo replications.
• Figure 4: Scenario 1: a predictive latent factor and a high-variance nuisance component. (a) Predictive performance (scaled test RMSE) of Hybrid PLS vs. PCR. For PCR, the x-axis indicates $L$ components per source (two functional, one scalar), yielding $3L$ total predictors. (b) Absolute correlations between PC scores from the three predictor sources. Boxplots show distributions across 100 replications.
• Figure 5: Scenario 2: one functional latent factor. (a) Predictive performance (scaled test RMSE) of Hybrid PLS vs. PCR. For PCR, the x-axis indicates $L$ components per source (two functional, one scalar), yielding $3L$ total predictors. (b) Absolute correlations between PC scores from the three predictor sources. Boxplots show distributions across 100 replications.

Theorems & Definitions (26)

• Definition 3.1: Hybrid Space
• Lemma 3.1
• Theorem 3.2
• Theorem 3.3: Properties of Population PLS
• Theorem 3.4
• Theorem 4.1: Regularized estimation of PLS component direction
• Theorem 4.2: Closed-form solution
• Lemma 4.3: Closed-form solution
• Lemma 4.4
• Definition 5.1: Roughness-sensitive inner product