Differential Machine Learning for Time Series Prediction
Akash Yadav, Eulalia Nualart
TL;DR
The paper tackles forecasting nonlinear, chaotic time series and real-world financial data. It introduces Diff-LSTM, which jointly models the original series and its differential using a shared LSTM cell and a differential-aware loss $\\mathcal{L} = \\mathcal{L}_{\\text{original}} + \\lambda \\cdot \\mathcal{L}_{\\text{differential}}$. Across Mackey-Glass, Lorenz, Rössler, and ACI-Finance datasets, Diff-LSTM outperforms traditional architectures such as RNNs, CNNs, BD-LSTM, and ED-LSTM in both short- and long-horizon forecasts. The work demonstrates that differential learning can improve prediction accuracy with limited data and provides an open-source implementation for reproducibility.
Abstract
Accurate time series prediction is challenging due to the inherent nonlinearity and sensitivity to initial conditions. We propose a novel approach that enhances neural network predictions through differential learning, which involves training models on both the original time series and its differential series. Specifically, we develop a differential long short-term memory (Diff-LSTM) network that uses a shared LSTM cell to simultaneously process both data streams, effectively capturing intrinsic patterns and temporal dynamics. Evaluated on the Mackey-Glass, Lorenz, and Rössler chaotic time series, as well as a real-world financial dataset from ACI Worldwide Inc., our results demonstrate that the Diff- LSTM network outperforms prevalent models such as recurrent neural networks, convolutional neural networks, and bidirectional and encoder-decoder LSTM networks in both short-term and long-term predictions. This framework offers a promising solution for enhancing time series prediction, even when comprehensive knowledge of the underlying dynamics of the time series is not fully available.