Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep State Space Recurrent Neural Networks for Time Series Forecasting

Hugo Inzirillo

TL;DR

This work addresses forecasting in volatile digital-asset time series by integrating econometric regime-switching with recurrent neural networks to form Deep State Space Models. It introduces a learnable switching mechanism that drives time-varying transition probabilities using covariates, and applies this mechanism to GRU, LSTM, and TKAN architectures, with TKAN-based switching (m-TKAN) delivering the strongest performance. The approach improves regime identification and predictive accuracy, yielding more balanced, risk-aware forecasts for cryptocurrency markets. The framework offers practical benefits for volatility-aware trading and risk management in emerging asset classes, while providing a flexible template for combining econometrics with deep learning in sequential data tasks.

Abstract

We explore various neural network architectures for modeling the dynamics of the cryptocurrency market. Traditional linear models often fall short in accurately capturing the unique and complex dynamics of this market. In contrast, Deep Neural Networks (DNNs) have demonstrated considerable proficiency in time series forecasting. This papers introduces novel neural network framework that blend the principles of econometric state space models with the dynamic capabilities of Recurrent Neural Networks (RNNs). We propose state space models using Long Short Term Memory (LSTM), Gated Residual Units (GRU) and Temporal Kolmogorov-Arnold Networks (TKANs). According to the results, TKANs, inspired by Kolmogorov-Arnold Networks (KANs) and LSTM, demonstrate promising outcomes.

Deep State Space Recurrent Neural Networks for Time Series Forecasting

TL;DR

This work addresses forecasting in volatile digital-asset time series by integrating econometric regime-switching with recurrent neural networks to form Deep State Space Models. It introduces a learnable switching mechanism that drives time-varying transition probabilities using covariates, and applies this mechanism to GRU, LSTM, and TKAN architectures, with TKAN-based switching (m-TKAN) delivering the strongest performance. The approach improves regime identification and predictive accuracy, yielding more balanced, risk-aware forecasts for cryptocurrency markets. The framework offers practical benefits for volatility-aware trading and risk management in emerging asset classes, while providing a flexible template for combining econometrics with deep learning in sequential data tasks.

Abstract

We explore various neural network architectures for modeling the dynamics of the cryptocurrency market. Traditional linear models often fall short in accurately capturing the unique and complex dynamics of this market. In contrast, Deep Neural Networks (DNNs) have demonstrated considerable proficiency in time series forecasting. This papers introduces novel neural network framework that blend the principles of econometric state space models with the dynamic capabilities of Recurrent Neural Networks (RNNs). We propose state space models using Long Short Term Memory (LSTM), Gated Residual Units (GRU) and Temporal Kolmogorov-Arnold Networks (TKANs). According to the results, TKANs, inspired by Kolmogorov-Arnold Networks (KANs) and LSTM, demonstrate promising outcomes.
Paper Structure (20 sections, 42 equations, 27 figures, 7 tables)

This paper contains 20 sections, 42 equations, 27 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Simple Regime Switching models with Time Varying Transition Probabilities
  3. Simple Regime Switching (2 states)
  4. Covariates
  5. Impact of Covariates
  6. The Evolution of Deep Learning Methods
  7. Feed Forward Networks (FNNs)
  8. Recurring Neural Networks (RNNs)
  9. Deep State Space Models
  10. Switching Mechanism
  11. Switching RNNs
  12. Learning Task
  13. Results
  14. Conclusion
  15. Appendix
  16. ...and 5 more sections

Figures (27)

  • Figure 1: Bitcoin cumulative sum of log returns. The shaded part of the figure represents the downward trends observed. We have defined a "bearish regime" as the period when the 20-day rolling average of the cumulative sum of log returns, shifted by 20 days, is lower than the current 20-day rolling average of the cumulative sum of log returns.
  • Figure 2: Smoothed Marginal Probabilities (basic MS model, 2 regimes)
  • Figure 3: Smoothed Marginal Probabilities
  • Figure 4: Smoothed Marginal Probabilities
  • Figure 5: MP neuron
  • ...and 22 more figures