Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast and Interpretable Autoregressive Estimation with Neural Network Backpropagation

Anaísa Lucena, Ana Martins, Armando J. Pinho, Sónia Gouveia

Abstract

Autoregressive (AR) models remain widely used in time series analysis due to their interpretability, but convencional parameter estimation methods can be computationally expensive and prone to convergence issues. This paper proposes a Neural Network (NN) formulation of AR estimation by embedding the autoregressive structure directly into a feedforward NN, enabling coefficient estimation through backpropagation while preserving interpretability. Simulation experiments on 125,000 synthetic AR(p) time series with short-term dependence (1 <= p <= 5) show that the proposed NN-based method consistently recovers model coefficients for all series, while Conditional Maximum Likelihood (CML) fails to converge in approximately 55% of cases. When both methods converge, estimation accuracy is comparable with negligible differences in relative error, R2 and, perplexity/likelihood. However, when CML fails, the NN-based approach still provides reliable estimates. In all cases, the NN estimator achieves substantial computational gains, reaching a median speedup of 12.6x and up to 34.2x for higher model orders. Overall, results demonstrate that gradient-descent NN optimization can provide a fast and efficient alternative for interpretable AR parameter estimation.

Fast and Interpretable Autoregressive Estimation with Neural Network Backpropagation

Abstract

Autoregressive (AR) models remain widely used in time series analysis due to their interpretability, but convencional parameter estimation methods can be computationally expensive and prone to convergence issues. This paper proposes a Neural Network (NN) formulation of AR estimation by embedding the autoregressive structure directly into a feedforward NN, enabling coefficient estimation through backpropagation while preserving interpretability. Simulation experiments on 125,000 synthetic AR(p) time series with short-term dependence (1 <= p <= 5) show that the proposed NN-based method consistently recovers model coefficients for all series, while Conditional Maximum Likelihood (CML) fails to converge in approximately 55% of cases. When both methods converge, estimation accuracy is comparable with negligible differences in relative error, R2 and, perplexity/likelihood. However, when CML fails, the NN-based approach still provides reliable estimates. In all cases, the NN estimator achieves substantial computational gains, reaching a median speedup of 12.6x and up to 34.2x for higher model orders. Overall, results demonstrate that gradient-descent NN optimization can provide a fast and efficient alternative for interpretable AR parameter estimation.
Paper Structure (15 sections, 16 equations, 7 figures, 2 tables, 3 algorithms)

This paper contains 15 sections, 16 equations, 7 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Methods
  3. Statistical formulation of AR models
  4. AR parameter estimation via CML
  5. NN-based AR parameter estimation
  6. Study design and performance evaluation
  7. Simulation setting
  8. Implementation details
  9. Performance metrics
  10. Results
  11. Convergence in CML and NN estimation
  12. Accuracy, precision and computational efficiency
  13. Scalability with model order
  14. Conclusion and Future Work

Figures (7)

  • Figure 1: Feedforward neural network representation for AR($p$) time series prediction using lagged inputs $(x_{t-1},\dots,x_{t-p})$. The inverse transformation $t^{-1}(\cdot)$ maps the network weights to AR coefficients while ensuring stationarity.
  • Figure 2: Boxplot of the distribution of the maximum absolute inverse root (YW), by AR order and CML status.
  • Figure 3: Maximum absolute inverse root of the initial YW estimates within the unit circle (cropped for visualization): (a) CML successful, (b) CML unsuccessful, and coefficients estimated from (c) NN and (d) CML, when both methods converge.
  • Figure 4: Pairwise comparison between CML and NN estimation: (a) computation time ratio (CML/NN), (b) relative error difference (CML$-$NN) of the estimated coefficients, (c) MSE difference (CML$-$NN), (d) perplexity difference (CML$-$NN).
  • Figure 5: Bland-Altman Plot for the MSE Cost Function.
  • ...and 2 more figures