Spectral Predictability as a Fast Reliability Indicator for Time Series Forecasting Model Selection

TL;DR

This work introduces spectral predictability $\Omega$ as a fast, interpretable metric to guide time-series forecasting model selection. By relating forecast difficulty to the concentration of the signal’s spectrum, the authors demonstrate that zero-shot TSFMs excel on high-$\Omega$ data while performance gaps narrow on low-$\Omega$ data, enabling a cost-effective, regime-aware deployment strategy. Across controlled experiments and large-scale GIFT-Eval analyses, $\Omega$ stratifies model performance and reveals practical guidance: use zero-shot models for high predictability, and lightweight baselines for low predictability, with careful consideration of non-stationarity and data length. The study also uncovers a disjunction between spectral predictability and dynamical chaos, and identifies a critical low-$\Omega$ regime where all models struggle, motivating future research into robust architectures for genuinely difficult forecasting tasks.

Abstract

Practitioners deploying time series forecasting models face a dilemma: exhaustively validating dozens of models is computationally prohibitive, yet choosing the wrong model risks poor performance. We show that spectral predictability~$Ω$ -- a simple signal processing metric -- systematically stratifies model family performance, enabling fast model selection. We conduct controlled experiments in four different domains, then further expand our analysis to 51 models and 28 datasets from the GIFT-Eval benchmark. We find that large time series foundation models (TSFMs) systematically outperform lightweight task-trained baselines when $Ω$ is high, while their advantage vanishes as $Ω$ drops. Computing $Ω$ takes seconds per dataset, enabling practitioners to quickly assess whether their data suits TSFM approaches or whether simpler, cheaper models suffice. We demonstrate that $Ω$ stratifies model performance predictably, offering a practical first-pass filter that reduces validation costs while highlighting the need for models that excel on genuinely difficult (low-$Ω$) problems rather than merely optimizing easy ones.

Figures (7)

• Figure 1: Spectral predictability systematically affects forecasting difficulty. Across synthetic and real-world domains, sMAPE declines as $\Omega$ increases. Error bars show 95% CIs across series. The clearest pattern emerges in synthetic data where $\Omega$ is directly controlled. Note that less data was available for PEMS and Fitbit, leading to sparser graphs. Also note that model classes have been slightly offset horizontally for visual clarity.
• Figure 2: Predictability-error relationship at scale. Across 28 datasets and 51 models, average error (sMAPE) declines with increasing spectral predictability $\Omega$. Each point represents an average of all models on one dataset. We fit an ordinary least squares line of best fit with 95$\%$ confidence interval for visualization.
• Figure 3: Predictability-error relationship with model types split out. The model type classes were taken from GIFT-Eval's classification aksu2024giftevalbenchmarkgeneraltime.
• Figure 4: Model Types Binned for Clarity. Only model types with more than 4 representative models were chosen to be represented here for robustness and visual clarity.
• Figure 5: Relationship with LLE. A heatmap showing LLE and Omega. A higher LLE corresponds to larger amounts of chaos in a sequence. This plot implies that some of the datasets which exhibit high $\Omega$ also have high chaos, which could explain why the sMAPE unexpectedly worsens.