Predicting the Future by Retrieving the Past

TL;DR

Predicting the Future by Retrieving the Past (PFRP) introduces a Global Memory Bank (GMB) to explicitly store and retrieve global historical patterns for univariate time series forecasting. The method employs Predictive Contrastive Learning to build compact, representative memory via $K$-medoids clustering and a retrieval mechanism that selects top-$k$ similar lookback sequences to generate global predictions, which are then dynamically fused with a local model's output through a confidence gate, an output gate, and a dynamic fusion module. Experiments on seven real-world datasets show consistent improvements over state-of-the-art univariate forecasters, with larger gains on highly periodic data and competitive efficiency relative to other retrieval-based approaches. Overall, PFRP provides a model-agnostic, interpretable, and scalable means to leverage global historical patterns for improved forecasting.

Abstract

Deep learning models such as MLP, Transformer, and TCN have achieved remarkable success in univariate time series forecasting, typically relying on sliding window samples from historical data for training. However, while these models implicitly compress historical information into their parameters during training, they are unable to explicitly and dynamically access this global knowledge during inference, relying only on the local context within the lookback window. This results in an underutilization of rich patterns from the global history. To bridge this gap, we propose Predicting the Future by Retrieving the Past (PFRP), a novel approach that explicitly integrates global historical data to enhance forecasting accuracy. Specifically, we construct a Global Memory Bank (GMB) to effectively store and manage global historical patterns. A retrieval mechanism is then employed to extract similar patterns from the GMB, enabling the generation of global predictions. By adaptively combining these global predictions with the outputs of any local prediction model, PFRP produces more accurate and interpretable forecasts. Extensive experiments conducted on seven real-world datasets demonstrate that PFRP significantly enhances the average performance of advanced univariate forecasting models by 8.4\%. Codes can be found in https://github.com/ddz16/PFRP.

Figures (11)

• Figure 1: (a) Time series often contain highly similar subsequences across different periods. (b) The GMB is constructed from historical sliding window samples, containing pairs of lookback window features and their corresponding prediction horizon sequences. (c) During inference, relevant patterns are retrieved from the GMB to generate global prediction, which are then dynamically fused with local prediction from any prediction model to yield the final result.
• Figure 2: Construction of GMB. (a) Predictive Contrastive Learning. Positive sample pairs are identified as those whose prediction horizon sequences exhibit the lowest MSE. PCL aims to pull the encoded lookback window sequences of positive pairs closer in feature space. (b) K-medoids Clustering. Retain only $K$ representative medoids in feature space to construct GMB, which stores both their lookback window features and corresponding prediction horizon sequences.
• Figure 3: Schematic diagram of PFRP. The diagram can be interpreted through the following key processes: (1) Orange arrows represent the retrieval process from GMB. (2) Blue arrows denote the weight modulation process. (3) Red arrows illustrate the output modulation process, which generates the global prediction. (4) The local prediction process is depicted at the bottom. (5) Green arrows indicate the fusion of global and local predictions based on dynamic weights. (6) The bottom-right section details the structures of the confidence gate and output gate.
• Figure 4: Top: Visualization of the prediction results for four baseline models, both with and without PFRP. Bottom: Visualization of the top 2 most relevant historical sequences retrieved by PFRP.
• Figure 5: Left: Performance and efficiency comparison with two RAG-based prediction methods on the Weather dataset. Right: Performance comparison of three large time-series models (with/without PFRP) on two datasets.