Parameter Estimation of Long Memory Stochastic Processes with Deep Neural Networks

TL;DR

This work harnessed efficient process generators to provide high-quality synthetic training data, enabling the training of scale-invariant 1D Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) models.

Abstract

We present a purely deep neural network-based approach for estimating long memory parameters of time series models that incorporate the phenomenon of long-range dependence. Parameters, such as the Hurst exponent, are critical in characterizing the long-range dependence, roughness, and self-similarity of stochastic processes. The accurate and fast estimation of these parameters holds significant importance across various scientific disciplines, including finance, physics, and engineering. We harnessed efficient process generators to provide high-quality synthetic training data, enabling the training of scale-invariant 1D Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) models. Our neural models outperform conventional statistical methods, even those augmented with neural networks. The precision, speed, consistency, and robustness of our estimators are demonstrated through experiments involving fractional Brownian motion (fBm), the Autoregressive Fractionally Integrated Moving Average (ARFIMA) process, and the fractional Ornstein-Uhlenbeck (fOU) process. We believe that our work will inspire further research in the field of stochastic process modeling and parameter estimation using deep learning techniques.

Figures (18)

• Figure 1: MSE losses of different fBm Hurst-estimators by sequence length on a log-log scale.
• Figure 2: Empirical bias and deviation of the different fBm estimators by Hurst value, measured on sequences of length 12800. The neural models were fine tuned until $n=12800$. The bias of the R/S estimator ranges from 0.125 to -0.075, and was truncated on the plot.
• Figure 3: Empirical bias and deviation of the $\text{ARFIMA}(0,d,0)$ estimators by $d$. Measured on sequences of length 12800.
• Figure 4: Scatterplots of $M_{\text{LSTM}}$ model inferences on cross-processes of length 12800. The fBm model was fine-tuned on sequences up to length 12800, and the $\text{ARFIMA}(0,d,0)$ model was trained on sequences of length 12800.
• Figure 5: The figure shows Hurst-estimates for the daily S&P 500 log-volatility, calculated from 15-minute log-returns. Estimates use 252-day (one year) sliding windows with 189-day overlaps.