TimeRAF: Retrieval-Augmented Foundation model for Zero-shot Time Series Forecasting

TL;DR

TimeRAF tackles zero-shot time series forecasting by augmenting a pre-trained TSFM with external knowledge via a learnable retriever and Channel Prompting. It constructs a multi-domain knowledge base of time series data and trains the retriever end-to-end, using forecaster feedback to align retrieval with forecasting utility. Channel Prompting fuses retrieved information into input embeddings, while the TSFM backbone remains frozen to preserve pre-trained knowledge. Experiments across six long-sequence datasets show TimeRAF achieves state-of-the-art or competitive zero-shot performance and often surpasses some full-shot baselines, highlighting the practical impact of retrieval-augmented forecasting in time series domains.

Abstract

Time series forecasting plays a crucial role in data mining, driving rapid advancements across numerous industries. With the emergence of large models, time series foundation models (TSFMs) have exhibited remarkable generalization capabilities, such as zero-shot learning, through large-scale pre-training. Meanwhile, Retrieval-Augmented Generation (RAG) methods have been widely employed to enhance the performance of foundation models on unseen data, allowing models to access to external knowledge. In this paper, we introduce TimeRAF, a Retrieval-Augmented Forecasting model that enhance zero-shot time series forecasting through retrieval-augmented techniques. We develop customized time series knowledge bases that are tailored to the specific forecasting tasks. TimeRAF employs an end-to-end learnable retriever to extract valuable information from the knowledge base. Additionally, we propose Channel Prompting for knowledge integration, which effectively extracts relevant information from the retrieved knowledge along the channel dimension. Extensive experiments demonstrate the effectiveness of our model, showing significant improvement across various domains and datasets.

Figures (7)

• Figure 1: Left: Time series foundation models (TSFMs), while capable of zero-shot forecasting, are limited by insufficient prior knowledge, resulting in constrained prediction accuracy. Right: By dynamically retrieving relevant information from an external knowledge base, our TimeRAF enhances prediction accuracy, leading to more precise zero-shot forecasting performance.
• Figure 2: Overview of TimeRAF: TimeRAF utilizes a retriever to dynamically retrieve relevant candidates from an external knowledge base and then utilizes the proposed Channel Prompting module to integrate knowledge between the retrieved data and the input. The knowledge-enhanced embeddings are subsequently fed into the backbone of the foundation model to improve forecasting results. During training, the backbone remains frozen.
• Figure 3: Improvement by TimeRAF on zero-shot forecasting. $5\%$ Few shot denotes finetuning TSFM with $5\%$ of downstream dataset. TimeRAF demonstrates significant improvements across various datasets, even outperforming results obtained by few-shot fine-tuning.
• Figure 4: Influence of knowledge base size. Smaller knowledge base provides less information, leading to worse performance.
• Figure 5: Influence of the Candidates Number $k$. As $k$ increases, the performance gradually improves due to the integration of more relevant knowledge. However, when $k$ exceeds a certain threshold, the abundance of information can introduce redundancy, negatively affecting the prediction.