Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Supervised Dynamic Dimension Reduction with Deep Neural Network

Zhanye Luo, Yuefeng Han, Xiufan Yu

TL;DR

The paper tackles forecasting with high-dimensional time-series predictors by learning supervised dynamic factors that are most predictive of $y_{t+h}$. It introduces Supervised Deep Dynamic PCA (SDDP), which constructs target-aware predictors $\hat{\mathbf{x}}_t^\star$ via per-predictor temporal DNNs and then applies PCA to obtain factors $\hat{\mathbf{g}}_t^\star$, which feed a nonlinear forecast $\mathcal{H}(\cdot)$. SDDP unifies diffusion-index forecasting and sufficient dimension reduction in a nonlinear, dynamic framework and extends to incomplete covariates with a mask-based approach and imputation. Empirical results on five public datasets show substantial accuracy gains over state-of-the-art methods across diverse forecasting architectures, with robustness to missing data and enhanced interpretability of the latent factors.

Abstract

This paper studies the problem of dimension reduction, tailored to improving time series forecasting with high-dimensional predictors. We propose a novel Supervised Deep Dynamic Principal component analysis (SDDP) framework that incorporates the target variable and lagged observations into the factor extraction process. Assisted by a temporal neural network, we construct target-aware predictors by scaling the original predictors in a supervised manner, with larger weights assigned to predictors with stronger forecasting power. A principal component analysis is then performed on the target-aware predictors to extract the estimated SDDP factors. This supervised factor extraction not only improves predictive accuracy in the downstream forecasting task but also yields more interpretable and target-specific latent factors. Building upon SDDP, we propose a factor-augmented nonlinear dynamic forecasting model that unifies a broad family of factor-model-based forecasting approaches. To further demonstrate the broader applicability of SDDP, we extend our studies to a more challenging scenario when the predictors are only partially observable. We validate the empirical performance of the proposed method on several real-world public datasets. The results show that our algorithm achieves notable improvements in forecasting accuracy compared to state-of-the-art methods.

Supervised Dynamic Dimension Reduction with Deep Neural Network

TL;DR

The paper tackles forecasting with high-dimensional time-series predictors by learning supervised dynamic factors that are most predictive of . It introduces Supervised Deep Dynamic PCA (SDDP), which constructs target-aware predictors via per-predictor temporal DNNs and then applies PCA to obtain factors , which feed a nonlinear forecast . SDDP unifies diffusion-index forecasting and sufficient dimension reduction in a nonlinear, dynamic framework and extends to incomplete covariates with a mask-based approach and imputation. Empirical results on five public datasets show substantial accuracy gains over state-of-the-art methods across diverse forecasting architectures, with robustness to missing data and enhanced interpretability of the latent factors.

Abstract

This paper studies the problem of dimension reduction, tailored to improving time series forecasting with high-dimensional predictors. We propose a novel Supervised Deep Dynamic Principal component analysis (SDDP) framework that incorporates the target variable and lagged observations into the factor extraction process. Assisted by a temporal neural network, we construct target-aware predictors by scaling the original predictors in a supervised manner, with larger weights assigned to predictors with stronger forecasting power. A principal component analysis is then performed on the target-aware predictors to extract the estimated SDDP factors. This supervised factor extraction not only improves predictive accuracy in the downstream forecasting task but also yields more interpretable and target-specific latent factors. Building upon SDDP, we propose a factor-augmented nonlinear dynamic forecasting model that unifies a broad family of factor-model-based forecasting approaches. To further demonstrate the broader applicability of SDDP, we extend our studies to a more challenging scenario when the predictors are only partially observable. We validate the empirical performance of the proposed method on several real-world public datasets. The results show that our algorithm achieves notable improvements in forecasting accuracy compared to state-of-the-art methods.

Paper Structure

This paper contains 22 sections, 2 theorems, 18 equations, 4 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Setup.
  3. Methodology
  4. Nonliner Factor Model
  5. Supervised Deep Dynamic PCA for Time Series
  6. Feature Reconstruction and Dimension Reduction Based on Deep Neural Network
  7. Target Time Series Regression Based on Supervised Factors
  8. Supervised Deep Dynamic PCA with Incomplete Data
  9. Empirical Studies
  10. Forecasting with SDDP.
  11. SDDP with Partially Observed Covariates.
  12. Discussion of Limitations and Extensions
  13. Acknowledgments
  14. Supplement to Empirical Studies
  15. Dataset Information
  16. ...and 7 more sections

Key Result

Proposition 1

Suppose Assumption asmp:linear holds. Assume $q$ and $K$ are fixed. Let $\mathrm{\bf g}_t^\star=(\mathrm{\bf f}_t^\top, \ldots, \mathrm{\bf f}_{t - q + 1}^\top)^\top$. Then there exist rotation matrices $R\in\mathbb{R}^{qK\times qK}$ such that

Figures (4)

  • Figure 1: An illustration of the proposed Supervised Deep Dynamic PCA (SDDP) algorithm. Inputs are observed predictors $X=(\mathrm{\bf x}_1,\ldots,\mathrm{\bf x}_T)\in\mathbb{R}^{N\times T}$, target $y=(y_1,\ldots,y_T)^\top\in\mathbb{R}^{T}$. Output is the estimated supervised dynamic factors $\widehat{G}^\star$.
  • Figure 2: (a) Cumulative normalized error for each method; (b) radar chart comparing the overall performance between SDDP methods and vanilla baselines in five aspects. Detailed values and the normalization procedure are provided in the Appendix.
  • Figure S.1: Relative improvements (in percentage) of SDDP, sdPCA, and PCA over the Vanilla baseline.
  • Figure S.2: Relative improvements (in percentage) of SDDP vs. Vanilla baselines in the presence of missing values.

Theorems & Definitions (2)

  • Proposition 1
  • Lemma 1