TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning

TL;DR

<TiRex> addresses the challenge of zero-shot forecasting across long and short horizons by introducing a decoder-only xLSTM backbone with Contiguous Patch Masking (CPM) and targeted data augmentations. It enables coherent multi-patch horizon predictions and probabilistic forecasts via nine quantiles and quantile loss, trained on a large, diverse corpus including Chronos, synthetic Gaussian-process data, and GiftEval pre-training data. The model achieves state-of-the-art zero-shot performance on GiftEval-ZS and Chronos-ZS benchmarks with a compact 35M-parameter footprint and fast inference, outperforming significantly larger transformer-based models. The work highlights the effectiveness of state-tracking LSTM variants for time-series in-context learning and provides ablations confirming the benefits of CPM, augmentation, and backbone design for long-horizon uncertainty propagation.

Abstract

In-context learning, the ability of large language models to perform tasks using only examples provided in the prompt, has recently been adapted for time series forecasting. This paradigm enables zero-shot prediction, where past values serve as context for forecasting future values, making powerful forecasting tools accessible to non-experts and increasing the performance when training data are scarce. Most existing zero-shot forecasting approaches rely on transformer architectures, which, despite their success in language, often fall short of expectations in time series forecasting, where recurrent models like LSTMs frequently have the edge. Conversely, while LSTMs are well-suited for time series modeling due to their state-tracking capabilities, they lack strong in-context learning abilities. We introduce TiRex that closes this gap by leveraging xLSTM, an enhanced LSTM with competitive in-context learning skills. Unlike transformers, state-space models, or parallelizable RNNs such as RWKV, TiRex retains state-tracking, a critical property for long-horizon forecasting. To further facilitate its state-tracking ability, we propose a training-time masking strategy called CPM. TiRex sets a new state of the art in zero-shot time series forecasting on the HuggingFace benchmarks GiftEval and Chronos-ZS, outperforming significantly larger models including TabPFN-TS (Prior Labs), Chronos Bolt (Amazon), TimesFM (Google), and Moirai (Salesforce) across both short- and long-term forecasts.

Figures (21)

• Figure 1: Two exemplary time series from the GiftEval benchmark. For both examples, we show one plot with the full context and TiRex's prediction, as well as zoomed-in forecasts of the best-performing zero-shot models. Each plot shows the ground truth signal in blue, the model's (median) prediction in orange, and the uncertainty bounds in gray. (a) A time series that exhibits strong peaks. Only TiRex is capable of predicting the periodic short spikes. (b) A time series with strong but noisy periodical behavior. TiRex predicts a meaningful uncertainty estimate (quantile range) over the long forecast horizon, while TimesFM and Chronos Bolt struggle because of collapsing quantiles.
• Figure 2: Architecture overview of TiRex. The model comprises two main components: the xLSTM blocks and a residual block in the input and output layers. The illustrated forecast shows the forecasted series is in blue and the forecast of TiRex in orange. During inference, only the last three output windows are of interest.
• Figure 3: Illustration of Contiguous Patch Masking and the different training augmentations.
• Figure 4: Results of the GiftEval-ZS benchmark: Aggregated scores of the overall benchmark and the short- and long-term performances. Additionally, the average rank in terms of CRPS, as in the public leaderboard, is presented. Lower values are better. "Zero-shot Leak" refers to models which are partly trained on the benchmark datasets (Overlap:: Moirai $19\%$, TimesFM $10\%$, TTM: $16\%$). We trained TiRex with 6 different seeds and report the observed standard deviation in the plot.
• Figure 5: Results of pre-trained model on the Chronos-ZS benchmark. The aggregated MASE and WQL scores, and the average rank in terms of WQL is shown. Lower values are better. "Zero-shot Leak" refers to models which are partly trained on the benchmark datasets (Overlap: Moirai $82\%$, TimesFM $15\%$, TTM: $11\%$).