Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

High-dimensional Autoregressive Modeling for Time Series with Hierarchical Structures

Lan Li, Shibo Yu, Yingzhou Wang, Guodong Li

Abstract

Modern applications have made ubiquitous high-dimensional data, especially time-dependent data, with more and more complicated structures, and it also has become more frequent to encounter the scenario of hierarchical relationships among variables. However, there is still a lack of supervised learning tool in the literature for them. To fill this gap, we introduce a new model-designing framework, and it then combines with unsupervised factor modeling tools to form an efficient and interpretable autoregressive model for high-dimensional time series with hierarchical structures. An ordinary least squares estimation is considered, and its non-asymptotic properties are established. Moreover, we propose an algorithm to search for estimates, and a boosting method is also suggested for hyperparameter selection. Simulation experiments are conducted to evaluate finite-sample performance of the proposed methodology, and its usefulness is demonstrated by an application to the Personality-120 dataset.

High-dimensional Autoregressive Modeling for Time Series with Hierarchical Structures

Abstract

Modern applications have made ubiquitous high-dimensional data, especially time-dependent data, with more and more complicated structures, and it also has become more frequent to encounter the scenario of hierarchical relationships among variables. However, there is still a lack of supervised learning tool in the literature for them. To fill this gap, we introduce a new model-designing framework, and it then combines with unsupervised factor modeling tools to form an efficient and interpretable autoregressive model for high-dimensional time series with hierarchical structures. An ordinary least squares estimation is considered, and its non-asymptotic properties are established. Moreover, we propose an algorithm to search for estimates, and a boosting method is also suggested for hyperparameter selection. Simulation experiments are conducted to evaluate finite-sample performance of the proposed methodology, and its usefulness is demonstrated by an application to the Personality-120 dataset.

Paper Structure

This paper contains 30 sections, 13 theorems, 142 equations, 6 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Model settings
  3. Tucker tensor regression and supervised factor modeling
  4. Purely hierarchical structure
  5. General hierarchical structure
  6. Autoregressive model for hierarchical time series
  7. High-dimensional estimation
  8. Ordinary least squares approach
  9. Non-asymptotic properties for tensor regression settings
  10. Non-asymptotic properties for tensor autoregressive models
  11. Algorithms and hyperparameter selection
  12. Algorithms
  13. Hyperparameter selection
  14. Simulation studies
  15. Real data analysis
  16. ...and 15 more sections

Key Result

Proposition 1

For any $\hbox{\boldmath$\mathscr{X}$}\in \mathbb{R}^{p_1 \times p_2 \times \cdots \times p_M}$, an action order $\alpha=(\alpha_{(1)},\alpha_{(2)},\ldots, \alpha_{(M)})$ is used for the permutation, and factor matrices $\{\bm{G}_{m}\in\mathbb{O}^{r_{m-1}p_{\alpha_{(m)}}\times r_{m}}\}_{m=1}^M$ with

Figures (6)

  • Figure 1: Three hierarchical feature extraction procedures for a second-order tensor $\hbox{\boldmath$\mathscr{X}$}_t\in \mathbb{R}^{p_1 \times p_2}$: hierarchical factor model (HFM), higher-order factor model (HOFM) and our model.
  • Figure 2: (a) Averaged MSEs under different action orders with varying rank $r_k$. (b) Averaged estimation errors $\|\hbox{\boldmath$\mathscr{\widehat{A}}$}_{\mathrm{AR}}-\hbox{\boldmath$\mathscr{A}$}^*_\mathrm{AR}\|_{\mathrm{F}}$ under three hyperparameter settings: (i) varying dimension of $q$, (ii) varying rank of $r$, and (iii) varying sample size of $T$. Three distributions of $\hbox{\boldmath$\mathscr{E}$}_t$ are considered, as specified in the legend.
  • Figure 3: Heatmaps of loading matrices for two predictor factors. The upper and lower panels correspond to action orders $\alpha_1=(3,2,1)$ and $\alpha_2=(1,3,2)$, respectively.
  • Figure 4: Heatmaps of reshaped estimated component matrices of the predictor factor for action order $\alpha_1=(3,2,1)$ at (a)--(c) and that of the estimated coefficient matrix of $\bm{\Theta}$ at (d).
  • Figure E.1: Average rolling forecast MSEs for HOFM, HFM, and our proposed method under our new supervised modeling framework.
  • ...and 1 more figures

Theorems & Definitions (30)

  • Remark 1
  • Remark 2
  • Proposition 1
  • Theorem 1
  • Theorem 2
  • Proposition 2
  • Proposition B.1
  • proof
  • Remark S.1
  • proof
  • ...and 20 more