Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TEFL: Prediction-Residual-Guided Rolling Forecasting for Multi-Horizon Time Series

Xiannan Huang, Shen Fang, Shuhan Qiu, Chengcheng Yu, Jiayuan Du, Chao Yang

TL;DR

TEFL (Temporal Error Feedback Learning), a unified learning framework that explicitly incorporates historical residuals from rolling forecasts into the forecasting pipeline during both training and evaluation, offers a simple, general, and effective enhancement to modern deep forecasting systems.

Abstract

Time series forecasting plays a critical role in domains such as transportation, energy, and meteorology. Despite their success, modern deep forecasting models are typically trained to minimize point-wise prediction loss without leveraging the rich information contained in past prediction residuals from rolling forecasts - residuals that reflect persistent biases, unmodeled patterns, or evolving dynamics. We propose TEFL (Temporal Error Feedback Learning), a unified learning framework that explicitly incorporates these historical residuals into the forecasting pipeline during both training and evaluation. To make this practical in deep multi-step settings, we address three key challenges: (1) selecting observable multi-step residuals under the partial observability of rolling forecasts, (2) integrating them through a lightweight low-rank adapter to preserve efficiency and prevent overfitting, and (3) designing a two-stage training procedure that jointly optimizes the base forecaster and error module. Extensive experiments across 10 real-world datasets and 5 backbone architectures show that TEFL consistently improves accuracy, reducing MAE by 5-10% on average. Moreover, it demonstrates strong robustness under abrupt changes and distribution shifts, with error reductions exceeding 10% (up to 19.5%) in challenging scenarios. By embedding residual-based feedback directly into the learning process, TEFL offers a simple, general, and effective enhancement to modern deep forecasting systems.

TEFL: Prediction-Residual-Guided Rolling Forecasting for Multi-Horizon Time Series

TL;DR

TEFL (Temporal Error Feedback Learning), a unified learning framework that explicitly incorporates historical residuals from rolling forecasts into the forecasting pipeline during both training and evaluation, offers a simple, general, and effective enhancement to modern deep forecasting systems.

Abstract

Time series forecasting plays a critical role in domains such as transportation, energy, and meteorology. Despite their success, modern deep forecasting models are typically trained to minimize point-wise prediction loss without leveraging the rich information contained in past prediction residuals from rolling forecasts - residuals that reflect persistent biases, unmodeled patterns, or evolving dynamics. We propose TEFL (Temporal Error Feedback Learning), a unified learning framework that explicitly incorporates these historical residuals into the forecasting pipeline during both training and evaluation. To make this practical in deep multi-step settings, we address three key challenges: (1) selecting observable multi-step residuals under the partial observability of rolling forecasts, (2) integrating them through a lightweight low-rank adapter to preserve efficiency and prevent overfitting, and (3) designing a two-stage training procedure that jointly optimizes the base forecaster and error module. Extensive experiments across 10 real-world datasets and 5 backbone architectures show that TEFL consistently improves accuracy, reducing MAE by 5-10% on average. Moreover, it demonstrates strong robustness under abrupt changes and distribution shifts, with error reductions exceeding 10% (up to 19.5%) in challenging scenarios. By embedding residual-based feedback directly into the learning process, TEFL offers a simple, general, and effective enhancement to modern deep forecasting systems.
Paper Structure (37 sections, 5 theorems, 49 equations, 7 figures, 10 tables, 1 algorithm)

This paper contains 37 sections, 5 theorems, 49 equations, 7 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Deep learning based time series prediction model
  4. Forecasting with historical prediction residuals
  5. Method
  6. Selection of historical prediction errors
  7. The design of error module
  8. Training process
  9. Phase 1: Warm-up with Temporally Structured Residuals
  10. Phase 2: Joint Training with Causal Error Simulation
  11. Theoretical Justification
  12. Experiment
  13. Set up
  14. Results
  15. Robust to Shocks
  16. ...and 22 more sections

Key Result

Proposition 4.1

Consider the state-space model with observation noise ( $\sigma_\varepsilon > 0$ ) and nonlinear dynamics such that $\mu_f' := \mathbb{E}_{x \sim \pi}[f'(x)] \neq 0$ . Then the prediction residuals of the optimal one-step forecaster satisfy Thus, even an oracle predictor leaves structured, predictable residuals whenever observations are noisy and the dynamics are nonlinear—a common condition in r

Figures (7)

  • Figure 1: Example of historical errors
  • Figure 2: Illustration of TEFL joint training pipeline.
  • Figure 3: Examples of shock and distribution shift
  • Figure 4: Comparison of Baseline and TEFL Methods for different input window size
  • Figure 5: Examples about spectral flatness
  • ...and 2 more figures

Theorems & Definitions (7)

  • Proposition 4.1: Lag-1 Autocovariance of Residuals
  • Theorem 4.2: Finite-Sample MSE Reduction
  • Proposition 1.1: Lag-1 Autocovariance of Residuals
  • proof
  • Theorem 1.2: Finite-Sample MSE Reduction by TEFL
  • proof
  • Theorem 1.3: Theorem 1 in merlevede2011bernstein