Retrieval Augmented Time Series Forecasting

TL;DR

RAFT addresses the challenge of forecasting nonstationary time series by injecting a retrieval-based inductive bias. It retrieves similar historical patches from the training data and leverages the subsequent values to augment a simple predictor, with multi-period downsampling to capture both short- and long-term patterns. Empirical results across ten benchmarks show consistent improvements over contemporary baselines, and analyses on synthetic data reveal that retrieval benefits are greatest for rare or weakly correlated patterns. The approach is also effective as a plug-in enhancement for transformer-based models, suggesting broad practical impact for real-world forecasting tasks.

Abstract

Time series forecasting uses historical data to predict future trends, leveraging the relationships between past observations and available features. In this paper, we propose RAFT, a retrieval-augmented time series forecasting method to provide sufficient inductive biases and complement the model's learning capacity. When forecasting the subsequent time frames, we directly retrieve historical data candidates from the training dataset with patterns most similar to the input, and utilize the future values of these candidates alongside the inputs to obtain predictions. This simple approach augments the model's capacity by externally providing information about past patterns via retrieval modules. Our empirical evaluations on ten benchmark datasets show that RAFT consistently outperforms contemporary baselines with an average win ratio of 86%.

Figures (9)

• Figure 1: Illustration of a motivating example of retrieval in time-series forecasting.
• Figure 2: Illustration of retrieval module architecture. First, we consider consecutive time frames from the entire time series $\mathbf{S}$ as key-value pairs and construct a candidate set using a sliding window approach. Given an input time series as the query, the retrieval module computes the similarity between the query and the keys in the candidate set that do not overlap temporally. Based on the similarity, the top-$m$ candidates are selected, and attention weights are calculated via SoftMax. The final result is obtained through a weighted sum of the corresponding values.
• Figure 3: Illustration of the proposed architecture, RAFT. The input time series $\mathbf{x}$ and the entire past observed time series $\mathbf{S}$ are first downsampled to generate multiple series with different periods. Then, a retrieval module is applied to each series to retrieve information relevant to the current input. The retrieved results are projected to the same dimension via a linear layer, and the results from different periods are summed to aggregate the information. Finally, the input time series is concatenated with the aggregated retrieved results, and a linear layer is applied to produce the final prediction.
• Figure 4: Analysis of the correlation between (a) the key similarity and value similarity, and (b) the value similarity and model performance changes measured by MSE (%). Each dot represents each input patch from the ETTh1 test dataset. Key similarity refers to the average similarity between input query ($\mathbf{x}$) and all retrieved key patches ($\mathcal{K}$). Value similarity refers to the average similarity between actual future value ($\mathbf{y}_0$) and all retrieved value patches ($\mathcal{V}$).
• Figure 5: Visualization of a synthetic time series with short-term patterns and the corresponding predictions over the rare short-term pattern from models with and without the retrieval module. MSE of predictions in this example without retrieval is 0.087, while with retrieval, it improves to 0.035.