Deep Time-series Forecasting Needs Kernelized Moment Balancing

TL;DR

This work reframes deep time-series forecasting as a distribution-balancing task and shows that existing objectives fail Imbens' criterion by only enforcing moment balance for a few functions. It introduces Direct Forecasting with Kernelized Moment Balancing (KMB-DF), which adaptively selects informative balancing functions from an RKHS and employs a soft-margin, kernel-based objective to ensure sufficient distribution alignment. The method is model-agnostic and demonstrates state-of-the-art forecasting accuracy across diverse datasets and backbones, with extensive ablations validating the contributions of kernel selection and soft-margin relaxation. The approach offers a principled way to mitigate autocorrelation bias and improve generalization, at minimal training overhead, making it practically impactful for large-scale time-series forecasting tasks.

Abstract

Deep time-series forecasting can be formulated as a distribution balancing problem aimed at aligning the distribution of the forecasts and ground truths. According to Imbens' criterion, true distribution balance requires matching the first moments with respect to any balancing function. We demonstrate that existing objectives fail to meet this criterion, as they enforce moment matching only for one or two predefined balancing functions, thus failing to achieve full distribution balance. To address this limitation, we propose direct forecasting with kernelized moment balancing (KMB-DF). Unlike existing objectives, KMB-DF adaptively selects the most informative balancing functions from a reproducing kernel hilbert space (RKHS) to enforce sufficient distribution balancing. We derive a tractable and differentiable objective that enables efficient estimation from empirical samples and seamless integration into gradient-based training pipelines. Extensive experiments across multiple models and datasets show that KMB-DF consistently improves forecasting accuracy and achieves state-of-the-art performance. Code is available at https://anonymous.4open.science/r/KMB-DF-403C.

Figures (5)

• Figure 1: The forecast sequence of MSE (in blue) and KMB-DF (in red), with historical length $\mathrm{H}=96$.
• Figure 2: Improvement of KMB-DF applied to different forecast models, shown with colored bars for means over forecast lengths (96, 192, 336, 720) and error bars for 50% confidence intervals.
• Figure 3: The forecast sequences generated with DF and KMB-DF. The forecast length is set to 192 and the experiment is conducted on ETTm1 and ETTh2.
• Figure 4: Performance of different forecast models with and without KMB-DF. The forecast errors are averaged over forecast lengths and the error bars represent 50% confidence intervals.
• Figure 5: Running time (ms) with varying forecast horizons.

Theorems & Definitions (11)

• Lemma 2.1: Autocorrelation bias
• Definition 3.1: Kernel function
• Definition 3.2: Universal kernel
• Theorem 3.3
• proof
• Lemma 2.1: Autocorrelation bias, Lemma \ref{['lem:bias']} in the main text
• proof
• Lemma 2.2: Representer theorem
• proof
• Theorem 2.3