Beyond Data Scarcity: A Frequency-Driven Framework for Zero-Shot Forecasting

TL;DR

The approach, Freq-Synth, improves the robustness of both foundation as well as nonfoundation forecast models in zero-shot and few-shot settings, facilitating more reliable time series forecasting under limited data scenarios.

Abstract

Time series forecasting is critical in numerous real-world applications, requiring accurate predictions of future values based on observed patterns. While traditional forecasting techniques work well in in-domain scenarios with ample data, they struggle when data is scarce or not available at all, motivating the emergence of zero-shot and few-shot learning settings. Recent advancements often leverage large-scale foundation models for such tasks, but these methods require extensive data and compute resources, and their performance may be hindered by ineffective learning from the available training set. This raises a fundamental question: What factors influence effective learning from data in time series forecasting? Toward addressing this, we propose using Fourier analysis to investigate how models learn from synthetic and real-world time series data. Our findings reveal that forecasters commonly suffer from poor learning from data with multiple frequencies and poor generalization to unseen frequencies, which impedes their predictive performance. To alleviate these issues, we present a novel synthetic data generation framework, designed to enhance real data or replace it completely by creating task-specific frequency information, requiring only the sampling rate of the target data. Our approach, Freq-Synth, improves the robustness of both foundation as well as nonfoundation forecast models in zero-shot and few-shot settings, facilitating more reliable time series forecasting under limited data scenarios.

Figures (12)

• Figure 1: We show an example of frequency confusion, where adding more frequencies gradually degrades performance (left). We also observe large performance differences when the fundamental target frequency exist vs. absent in the train set, implying poor frequency generalization (right).
• Figure 2: Transfer learning performance bars for various frequency-based alignments between the train and test sets. Ranged values represent the periodogram PCC (left), ETT is a single dataset with different sampling rates (middle), and the remaining are sector based categories (right).
• Figure 3: We construct a pool of sine waves that are harmonics to a given fundamental frequency (left). We create a multivariate time series by sampling and adding sines per variate (right).
• Figure 4: Zero-shot performance of pre-trained models on signals with one and two harmonics (top and bottom, respectively). The models perform well on the 1/24 and 1/12 frequencies, but for the remaining frequencies, the performance decreases significantly.
• Figure 5: The influence of the number of harmonics on the ZS performance per dataset, where each result reports the average MSE for TTM, GPT4TS, PatchTST, UniTime, and Moment for a forecast horizon 96.

Theorems & Definitions (2)

• Definition 4.1: Frequency Confusion
• Definition 4.2: Frequency Generalization