Incorporating Taylor Series and Recursive Structure in Neural Networks for Time Series Prediction

TL;DR

The paper addresses univariate time-series forecasting by introducing TaylorNet, a neural architecture that fuses ResNet-style residual connections with a Taylor-series framework to model temporal dynamics. TaylorNet uses derivatives estimated by neural sub-networks to predict the next value, with second- and third-order variants TaylorNet2 and TaylorNet3 expressed as $\hat{x}_{i+d} = x_{i+d-1} + \Delta t\mathcal{N}_1(...) + \frac{\Delta t^2}{2}\mathcal{N}_2(...)$ and optionally $+ \frac{\Delta t^3}{6}\mathcal{N}_3(...)$, respectively; a CNN front-end processes the past $d-1$ observations. The authors further extend the approach with Recursive TaylorNet, which performs multiple smaller steps per prediction via $m$ substeps, leading to Recursive TaylorNet2 (and Recursive TaylorNet3) that improve accuracy on many datasets. Empirical results over 18 diverse univariate datasets show TaylorNet2 achieving the best average rank among baselines (CNN, ResNet, LSTM), while recursive variants (notably R Taylor 2) often yield the strongest overall performance. The work highlights the potential of integrating higher-order derivative information and recursion into neural time-series models, offering a promising direction for more accurate forecasting in finance, climate, biology, and beyond.

Abstract

Time series analysis is relevant in various disciplines such as physics, biology, chemistry, and finance. In this paper, we present a novel neural network architecture that integrates elements from ResNet structures, while introducing the innovative incorporation of the Taylor series framework. This approach demonstrates notable enhancements in test accuracy across many of the baseline datasets investigated. Furthermore, we extend our method to incorporate a recursive step, which leads to even further improvements in test accuracy. Our findings underscore the potential of our proposed model to significantly advance time series analysis methodologies, offering promising avenues for future research and application.

Figures (2)

• Figure 1: Example of a TaylorNet2 architecture with an input sequence length of 3 and a single hidden layer. The outputs $\mathcal{N}_1$ and $\mathcal{N}_2$ approximate the derivatives $\frac{dx}{dt}$ and $\frac{d^2x}{dt^2}$, respectively, of the underlying dynamics of the time series.
• Figure 2: Example of a TaylorNet3 archietcture with an input sequence length of 3 and a single hidden layer. The outputs $\mathcal{N}_1$, $\mathcal{N}_2$, and $\mathcal{N}_3$ approximate the derivatives $\frac{dx}{dt}$, $\frac{d^x}{dt^2}$, and $\frac{d^3x}{dt^3}$, respectively, of the underlying dynamics of the time series.