Stratify: Unifying Multi-Step Forecasting Strategies

TL;DR

Stratify presents a unified, parameterised space for multi-step forecasting strategies by extending RectifyMO into a general framework that indices base and rectifier components with a ρ-δ-ι triple. Across 18 benchmarks, horizons 10–80, and five function classes, novel Stratify strategies consistently outperform existing methods, with notable gains for RNN/LSTM models. The approach reveals task-dependent strategy effectiveness and enables efficient exploration of the strategy space, offering practical guidance for selecting forecasting strategies. While comprehensive, the study focuses on univariate time series, suggesting multivariate extensions and meta-learning to map dataset features to optimal Stratify configurations as fruitful future work.

Abstract

A key aspect of temporal domains is the ability to make predictions multiple time steps into the future, a process known as multi-step forecasting (MSF). At the core of this process is selecting a forecasting strategy, however, with no existing frameworks to map out the space of strategies, practitioners are left with ad-hoc methods for strategy selection. In this work, we propose Stratify, a parameterised framework that addresses multi-step forecasting, unifying existing strategies and introducing novel, improved strategies. We evaluate Stratify on 18 benchmark datasets, five function classes, and short to long forecast horizons (10, 20, 40, 80). In over 84% of 1080 experiments, novel strategies in Stratify improved performance compared to all existing ones. Importantly, we find that no single strategy consistently outperforms others in all task settings, highlighting the need for practitioners explore the Stratify space to carefully search and select forecasting strategies based on task-specific requirements. Our results are the most comprehensive benchmarking of known and novel forecasting strategies. We make code available to reproduce our results.

Figures (15)

• Figure 1: Relative MSE over 18 Benchmark Datasets and multistep horizons 10, 20, 40, 80 for five function classes. Exploring Stratify's novel strategies consistently outperform existing strategies within the unified space. The best performing novel strategy is compared to the best performing existing strategy, both of which are accessible in the Stratify space. Consistent relative MSE below the dashed line show that exploring Stratify is beneficial across short to long horizons.
• Figure 2: Summary of the strategies in MSF. In bold are our contributions. We extend the single-output Rectify strategy into its multi-output taieb2010multiple variant, analogous to RecMO, DirMO taieb2014machine, and DirRecMO noa2024dirrecmo. Stratify is a framework which generalises all existing strategies and introduces novel strategies with improved performance. Lines show the evolution and fusion of previous strategies to form new ones.
• Figure 3: An example of Rectify, the recurisve forecast captures the majority of the dynamics and the variance in residuals is modelled effectively by the direct forecast.
• Figure 4: We show Figure 3 from noa2024dirrecmo. Forecasts over $H = 24$ for each forecasting strategy are shown, where they are parameterised by $k$, instead of $\sigma$ as used in this work. Recursive/RecMO strategies consist of a single model that iterates forwards $k$ values using its own predictions as future inputs until the horizon is reached. Direct methods train an independent model for each $k$ values. DirRec methods also train independent models for each $k$ values but use the previous predictions as input into the subsequent prediction. All methods become equivalent when $k=H$.
• Figure 5: