Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fitting ARMA Time Series Models without Identification: A Proximal Approach

Yin Liu, Sam Davanloo Tajbakhsh

Abstract

Fitting autoregressive moving average (ARMA) time series models requires model identification before parameter estimation. Model identification involves determining the order of the autoregressive and moving average components which is generally performed by inspection of the autocorrelation and partial autocorrelation functions or other offline methods. In this work, we regularize the parameter estimation optimization problem with a non-smooth hierarchical sparsity-inducing penalty based on two path graphs that allow performing model identification and parameter estimation simultaneously. A proximal block coordinate descent algorithm is then proposed to solve the underlying optimization problem efficiently. The resulting model satisfies the required stationarity and invertibility conditions for ARMA models. Numerical results supporting the proposed method are also presented.

Fitting ARMA Time Series Models without Identification: A Proximal Approach

Abstract

Fitting autoregressive moving average (ARMA) time series models requires model identification before parameter estimation. Model identification involves determining the order of the autoregressive and moving average components which is generally performed by inspection of the autocorrelation and partial autocorrelation functions or other offline methods. In this work, we regularize the parameter estimation optimization problem with a non-smooth hierarchical sparsity-inducing penalty based on two path graphs that allow performing model identification and parameter estimation simultaneously. A proximal block coordinate descent algorithm is then proposed to solve the underlying optimization problem efficiently. The resulting model satisfies the required stationarity and invertibility conditions for ARMA models. Numerical results supporting the proposed method are also presented.

Paper Structure

This paper contains 18 sections, 2 theorems, 15 equations, 6 figures, 1 table, 2 algorithms.

Table of Contents

  1. INTRODUCTION
  2. CONTRIBUTIONS
  3. Related Work
  4. Notations
  5. PROBLEM DEFINITION
  6. PROPOSED METHOD
  7. Hierarchical Sparsity for ARMA Models
  8. Latent Overlapping Group (LOG) Lasso
  9. The Proposed Hierarchically Sparse Learning Problem
  10. A Note on the Optimization Problem (HS-ARMA)
  11. NUMERICAL STUDIES
  12. Synthetic Data Generation Process
  13. Model Identification and Parameter Estimation Accuracy
  14. Prediction Performance
  15. Comparison with Other Methods
  16. ...and 3 more sections

Key Result

Lemma 3.1

The proximal operator of the LOG penalty defined over the ARMA DAG is separable, i.e., $\mathbf{prox}_{\Omega_{\text{LOG}}}(\mathbf{b}_1,\mathbf{b}_2)=( \mathbf{prox}_{\Omega_{\text{LOG}}^{\text{AR}}}(\mathbf{b}_1),\mathbf{prox}_{\Omega_{\text{LOG}}^{\text{MA}}}(\mathbf{b}_2) )$.

Figures (6)

  • Figure 1: Path graphs showing hierarchical sparsities: (Top) A graph with a variable per node for ${\boldsymbol{\beta}}\in\mathbb{R}^3$. (Bottom) A graph with multiple variables per node for ${\boldsymbol{\beta}}\in\mathbb{R}^5$.
  • Figure 2: DAG for the $\text{ARMA}(\bar{p},\bar{q})$ process. The red dotted rectangles illustrate the ascending grouping scheme for the LOG penalty.
  • Figure 3: The estimation error of HS-ARMA and two pre-identified models. The three thicker lines are the mean estimation errors and the thinner lines represent estimation errors for each sample.
  • Figure 4: Prediction RMSEs for the HS-ARMA method vs. the correctly and incorrectly identified models. Each grey thin line is the RMSE of HS-ARMA with one $\lambda_0$ from $\{0.5,1,2,3,5\}$ from ten realizations and the black thick line is the average of the grey lines. The green and red lines are the RMSEs from the ten realizations for correctly and incorrectly identified models.
  • Figure 5: Comparison of the proposed HS-ARMA method with the hierarchical lag (H-Lag) and $\ell_1$ penalty methods from wilms2017sparse.
  • ...and 1 more figures

Theorems & Definitions (4)

  • Lemma 3.1
  • proof
  • Lemma 3.2
  • proof