Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Principal component-guided sparse reduced-rank regression

Kanji Goto, Shintaro Yuki, Kensuke Tanioka, Hiroshi Yadohisa

TL;DR

The paper addresses multivariate regression with correlated responses and high-dimensional, grouped predictors by integrating principal component-guided regularization into reduced-rank regression. The model uses B = CD^T with a rank constraint and introduces group-aware, pcLasso-like penalties to bias coefficients toward high-variance PCs while accounting for within-group correlations, solved via an alternating least squares algorithm. Through extensive simulations and a real gene-expression dataset, the method demonstrates improved predictive accuracy and stability (lower MSE_Y and MSPE) relative to competitors, particularly under small-sample and grouped-variable settings. This approach offers a principled framework for leveraging correlation structures in both responses and explanatory groups to enhance predictive performance in complex multivariate problems.

Abstract

Reduced-rank regression estimates regression coefficients by imposing a low-rank constraint on the matrix of regression coefficients, thereby accounting for correlations among response variables. To further improve predictive accuracy and model interpretability, several regularized reduced-rank regression methods have been proposed. However, these existing methods cannot bias the regression coefficients toward the leading principal component directions while accounting for the correlation structure among explanatory variables. In addition, when the explanatory variables exhibit a group structure, the correlation structure within each group cannot be adequately incorporated.To address these limitations, we propose a new method that introduces pcLasso into the reduced-rank regression framework. The proposed method improves predictive accuracy by accounting for the correlation among response variables while strongly biasing the matrix of regression coefficients toward principal component directions with large variance. Furthermore, even in settings where the explanatory variables possess a group structure, the proposed method is capable of explicitly incorporating this structure into the estimation process. Finally, we illustrate the effectiveness of the proposed method through numerical simulations and real data application.

Principal component-guided sparse reduced-rank regression

TL;DR

The paper addresses multivariate regression with correlated responses and high-dimensional, grouped predictors by integrating principal component-guided regularization into reduced-rank regression. The model uses B = CD^T with a rank constraint and introduces group-aware, pcLasso-like penalties to bias coefficients toward high-variance PCs while accounting for within-group correlations, solved via an alternating least squares algorithm. Through extensive simulations and a real gene-expression dataset, the method demonstrates improved predictive accuracy and stability (lower MSE_Y and MSPE) relative to competitors, particularly under small-sample and grouped-variable settings. This approach offers a principled framework for leveraging correlation structures in both responses and explanatory groups to enhance predictive performance in complex multivariate problems.

Abstract

Reduced-rank regression estimates regression coefficients by imposing a low-rank constraint on the matrix of regression coefficients, thereby accounting for correlations among response variables. To further improve predictive accuracy and model interpretability, several regularized reduced-rank regression methods have been proposed. However, these existing methods cannot bias the regression coefficients toward the leading principal component directions while accounting for the correlation structure among explanatory variables. In addition, when the explanatory variables exhibit a group structure, the correlation structure within each group cannot be adequately incorporated.To address these limitations, we propose a new method that introduces pcLasso into the reduced-rank regression framework. The proposed method improves predictive accuracy by accounting for the correlation among response variables while strongly biasing the matrix of regression coefficients toward principal component directions with large variance. Furthermore, even in settings where the explanatory variables possess a group structure, the proposed method is capable of explicitly incorporating this structure into the estimation process. Finally, we illustrate the effectiveness of the proposed method through numerical simulations and real data application.
Paper Structure (12 sections, 2 theorems, 31 equations, 5 tables, 1 algorithm)

This paper contains 12 sections, 2 theorems, 31 equations, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Principal component-guided sparse reduced-rank regression model
  3. Model formula
  4. Objective function
  5. Estimation algorithm
  6. Numerical Study
  7. Simulation design
  8. Simulation results
  9. Results of the numerical simulations in the case without a group structure
  10. Results of the numerical simulations in the case with a group structure
  11. Real data application
  12. Conclusions

Key Result

Proposition 1

Fixing $\bm{D}$ as $\bm{D}^{*}$, the $i$-th row vector $\hat{\bm{C}}_{i}^{(k)} \in \mathbb{R}^{1 \times r}$$(i = 1, \ldots, p_k)$ of the estimator $\hat{\bm{C}}^{(k)}$ is updated in order to minimize Eq.object_group by the subgradient method friedman2007pathwise. where, $\bm{x}^{(k)}_{i} \in \mathbb{R}^{n \times 1}$ denote the $i$-th column of the explanatory variable matrix $\bm{X}^{(k)}$ for gr

Theorems & Definitions (4)

  • Proposition 1
  • proof
  • Proposition 2
  • proof