From Observations to States: Latent Time Series Forecasting

TL;DR

This paper addresses Latent Chaos in time series forecasting, where strong observation-space accuracy coexists with disordered latent dynamics. It proposes LatentTSF, a two-stage paradigm that expands observations into a high-dimensional latent state space with a frozen AutoEncoder and learns latent-state dynamics that are then decoded for forecasts. The training objective jointly optimizes latent-prediction and latent-alignment losses, with theory linking these losses to mutual information terms $I(oldsymbol{Z}_Y;oldsymbol{ ilde{Z}}_Y)$ and $I(oldsymbol{Y};oldsymbol{ ilde{Z}}_Y)$ to justify informative latent representations. Empirical results across six benchmarks show that LatentTSF improves both representation quality and forecasting accuracy, particularly for long horizons and high-dimensional data, and it remains compatible with a range of backbones. The work provides a principled, practical route to learning temporally coherent latent dynamics under partial observability, with publicly available code.

Abstract

Deep learning has achieved strong performance in Time Series Forecasting (TSF). However, we identify a critical representation paradox, termed Latent Chaos: models with accurate predictions often learn latent representations that are temporally disordered and lack continuity. We attribute this phenomenon to the dominant observation-space forecasting paradigm. Most TSF models minimize point-wise errors on noisy and partially observed data, which encourages shortcut solutions instead of the recovery of underlying system dynamics. To address this issue, we propose Latent Time Series Forecasting (LatentTSF), a novel paradigm that shifts TSF from observation regression to latent state prediction. Specifically, LatentTSF employs an AutoEncoder to project observations at each time step into a higher-dimensional latent state space. This expanded representation aims to capture underlying system variables and impose a smoother temporal structure. Forecasting is then performed entirely in the latent space, allowing the model to focus on learning structured temporal dynamics. Theoretical analysis demonstrates that our proposed latent objectives implicitly maximize mutual information between predicted latent states and ground-truth states and observations. Extensive experiments on widely-used benchmarks confirm that LatentTSF effectively mitigates latent chaos, achieving superior performance. Our code is available in https://github.com/Muyiiiii/LatentTSF.

Figures (8)

• Figure 1: Latent Chaos visualization under LatentTSF. Electricity dataset: multi-view comparison of (a) raw observations, (b) standard iTransformer embeddings, and (c) iTransformer embeddings trained with LatentTSF, shown at 0%/50%/100% training progress. Top: t-SNE visualizations (colored by time index). Bottom: frequency-domain spectra of the corresponding representations. Green box numbers report the mean normalized Euclidean distance between adjacent time steps, while Brown box numbers indicate forecasting MAE. Additional results are provided in App. \ref{['app:Latent_Chaos_Analysis']}.
• Figure 2: Overview of LatentTSF. The framework consists of a two-stage pipeline: (1) latent state space construction via a pre-trained AutoEncoder, and (2) latent states forecasting using a TSF backbone, followed by decoding to the observation space.
• Figure 3: Loss weights sensitivity of LatentTSF. MAE curves on ETTh1, ETTm1, and Electricity when varying the perceptual, prediction ($\mathcal{L}_{\text{Pred}}$), and alignment ($\mathcal{L}_{\text{Align}}$) weights for three backbones (iTransformer, CMoS, and DLinear).
• Figure 4: Interaction between $\mathcal{L}_{\text{Pred}}$ and $\mathcal{L}_{\text{Align}}$. MAE heatmaps over Pred/Align weight grids on three datasets (rows) and three backbones (columns). Stars mark the best MAE in each subplot.
• Figure 5: Latent structure of TimeBase on Electricity. t-SNE (top) and Fourier spectra (bottom) of decoder-pre embeddings from standard training vs. LatentTSF.