STAN: Smooth Transition Autoregressive Networks

TL;DR

This work introduces STAN, a neural network architecture that mirrors Smooth Transition Autoregressive (STAR) models by embedding learnable smooth transition functions within each layer to capture regime-switching nonlinear dynamics in time series. By keeping a feed-forward structure, STAN aims to combine the interpretability of econometric regime-switching with the flexibility and scalability of neural networks. Empirical evaluation on the PJM Hourly Energy dataset shows STAN delivers superior short-term forecasts and competitive long-term performance, with training efficiency akin to MLPs and faster than recurrent models. The study highlights STAN as a promising bridge between econometric theory and deep learning, with potential extensions to multivariate settings and interpretability analyses of learned transitions.

Abstract

Traditional Smooth Transition Autoregressive (STAR) models offer an effective way to model these dynamics through smooth regime changes based on specific transition variables. In this paper, we propose a novel approach by drawing an analogy between STAR models and a multilayer neural network architecture. Our proposed neural network architecture mimics the STAR framework, employing multiple layers to simulate the smooth transition between regimes and capturing complex, nonlinear relationships. The network's hidden layers and activation functions are structured to replicate the gradual switching behavior typical of STAR models, allowing for a more flexible and scalable approach to regime-dependent modeling. This research suggests that neural networks can provide a powerful alternative to STAR models, with the potential to enhance predictive accuracy in economic and financial forecasting.