Time-Series Forecasting for Out-of-Distribution Generalization Using Invariant Learning

TL;DR

This work tackles the challenge of out-of-distribution generalization in time-series forecasting by integrating invariant learning with an environment-inference mechanism. The proposed FOIL framework decomposes the target into a sufficiently predictable component, infers temporal environments via an EM-based multi-head arrangement, and trains an invariant representation across inferred environments using a surrogate loss and variance-regularized objective. Empirical results across multiple datasets and backbones show FOIL yields substantial gains over both standard TSF models and existing distribution-shift methods, with notable improvements in short-horizon forecasts and on datasets with strong OOD shifts. By preserving temporal adjacency and enabling model-agnostic integration, FOIL offers a practical pathway to more robust TSF under real-world distribution dynamics, while also suggesting avenues for fairness and causal analysis in future work.

Abstract

Time-series forecasting (TSF) finds broad applications in real-world scenarios. Due to the dynamic nature of time-series data, it is crucial to equip TSF models with out-of-distribution (OOD) generalization abilities, as historical training data and future test data can have different distributions. In this paper, we aim to alleviate the inherent OOD problem in TSF via invariant learning. We identify fundamental challenges of invariant learning for TSF. First, the target variables in TSF may not be sufficiently determined by the input due to unobserved core variables in TSF, breaking the conventional assumption of invariant learning. Second, time-series datasets lack adequate environment labels, while existing environmental inference methods are not suitable for TSF. To address these challenges, we propose FOIL, a model-agnostic framework that enables timeseries Forecasting for Out-of-distribution generalization via Invariant Learning. FOIL employs a novel surrogate loss to mitigate the impact of unobserved variables. Further, FOIL implements a joint optimization by alternately inferring environments effectively with a multi-head network while preserving the temporal adjacency structure, and learning invariant representations across inferred environments for OOD generalized TSF. We demonstrate that the proposed FOIL significantly improves the performance of various TSF models, achieving gains of up to 85%.

Figures (3)

• Figure 1: The structural causal model (SCM) for (a) existing invariant learning methods and (b) our proposed method. The key difference is that our method targets the sufficiently predictable part of the target, i.e., $\bm{\bm{Y}^{\text{suf}}}$ rather than the raw $\bm{Y}$, thus making invariant learning feasible.
• Figure 2: The overall framework of our proposed FOIL.
• Figure 3: Results of analytical experiments.