PDETime: Rethinking Long-Term Multivariate Time Series Forecasting from the perspective of partial differential equations

TL;DR

This paper presents PDETime, a novel LMTF model inspired by the principles of Neural PDE solvers, following the encoding-integration-decoding operations, which adapts effectively to the intrinsic spatiotemporal nature of the data and sets new benchmarks, achieving state-of-the-art results.

Abstract

Recent advancements in deep learning have led to the development of various models for long-term multivariate time-series forecasting (LMTF), many of which have shown promising results. Generally, the focus has been on historical-value-based models, which rely on past observations to predict future series. Notably, a new trend has emerged with time-index-based models, offering a more nuanced understanding of the continuous dynamics underlying time series. Unlike these two types of models that aggregate the information of spatial domains or temporal domains, in this paper, we consider multivariate time series as spatiotemporal data regularly sampled from a continuous dynamical system, which can be represented by partial differential equations (PDEs), with the spatial domain being fixed. Building on this perspective, we present PDETime, a novel LMTF model inspired by the principles of Neural PDE solvers, following the encoding-integration-decoding operations. Our extensive experimentation across seven diverse real-world LMTF datasets reveals that PDETime not only adapts effectively to the intrinsic spatiotemporal nature of the data but also sets new benchmarks, achieving state-of-the-art results

Figures (5)

• Figure 1: The framework of proposed PDETime which consists of an Encoder $E_{\theta}$, a Solver, and a Decoder $D_{\phi}$. Given the initial condition $x_{\tau_{0}}$, PDETime first simulates $\frac{\partial u(x,t_{\tau})}{\partial t_{\tau}}$ at each time step $t_{\tau}$ using the Encoder $E_{\theta}(x_{his},t_{\tau}, \tau)$; then uses the Solver to compute $\int_{\tau_{0}}^{\tau} \frac{\partial u(x,t_{\tau})}{\partial t_{\tau}} dt_{\tau}$, which is a numerical solver; finally, the Decoder maps integral term $z_{\tau}$ from latent space to special domains and predict the final results $\hat{x}_{\tau}=x_{\tau_{0}}+D_{\phi}(z_{\tau})$.
• Figure 2: The architecture of the Encoder which is used to simulate $\frac{\partial u(x,t_{\tau})}{\partial t_{\tau}}$. In this work, we use INRs to represent $x_{his}$, $t_{\tau}$, and $\tau$, and then use the Attention mechanism and linear mapping to aggregate the information of spatial domains and temporal domains.
• Figure 3: The framework of standard and variant of numerical solver proposed in our work.
• Figure 4: Evaluation on hyper-parameter impact. (a) MSE against hyper-parameter layers of INRs $k$ in Forecaster on ETTh1. (b) MSE against hyper-parameter layers of aggregation module $N$ in Forecaster on ETTh1. (c) MSE against hyper-parameter patch length $S$ in Estimator on ETTh1.
• Figure 5: Visualization of PDETime's prediction results on Traffic with different prediction length.