Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Closing the Loop: A Control-Theoretic Framework for Provably Stable Time Series Forecasting with LLMs

Xingyu Zhang, Hanyun Du, Zeen Song, Jianqi Zhang, Changwen Zheng, Wenwen Qiang

Abstract

Large Language Models (LLMs) have recently shown exceptional potential in time series forecasting, leveraging their inherent sequential reasoning capabilities to model complex temporal dynamics. However, existing approaches typically employ a naive autoregressive generation strategy. We identify a critical theoretical flaw in this paradigm: during inference, the model operates in an open-loop manner, consuming its own generated outputs recursively. This leads to inevitable error accumulation (exposure bias), where minor early deviations cascade into significant trajectory drift over long horizons. In this paper, we reformulate autoregressive forecasting through the lens of control theory, proposing \textbf{F-LLM} (Feedback-driven LLM), a novel closed-loop framework. Unlike standard methods that passively propagate errors, F-LLM actively stabilizes the trajectory via a learnable residual estimator (Observer) and a feedback controller. Furthermore, we provide a theoretical guarantee that our closed-loop mechanism ensures uniformly bounded error, provided the base model satisfies a local Lipschitz constraint. Extensive experiments demonstrate that F-LLM significantly mitigates error propagation, achieving good performance on time series benchmarks.

Closing the Loop: A Control-Theoretic Framework for Provably Stable Time Series Forecasting with LLMs

Abstract

Large Language Models (LLMs) have recently shown exceptional potential in time series forecasting, leveraging their inherent sequential reasoning capabilities to model complex temporal dynamics. However, existing approaches typically employ a naive autoregressive generation strategy. We identify a critical theoretical flaw in this paradigm: during inference, the model operates in an open-loop manner, consuming its own generated outputs recursively. This leads to inevitable error accumulation (exposure bias), where minor early deviations cascade into significant trajectory drift over long horizons. In this paper, we reformulate autoregressive forecasting through the lens of control theory, proposing \textbf{F-LLM} (Feedback-driven LLM), a novel closed-loop framework. Unlike standard methods that passively propagate errors, F-LLM actively stabilizes the trajectory via a learnable residual estimator (Observer) and a feedback controller. Furthermore, we provide a theoretical guarantee that our closed-loop mechanism ensures uniformly bounded error, provided the base model satisfies a local Lipschitz constraint. Extensive experiments demonstrate that F-LLM significantly mitigates error propagation, achieving good performance on time series benchmarks.
Paper Structure (38 sections, 2 theorems, 16 equations, 8 figures, 8 tables)

This paper contains 38 sections, 2 theorems, 16 equations, 8 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Notation and Problem Formulation
  4. TSF via Autoregression.
  5. The Open-Loop Vulnerability
  6. Theoretical Analysis
  7. Open-Loop Error Dynamics
  8. Closed-Loop Stabilization
  9. Stability Guarantee
  10. Deriving the Architecture from Theory
  11. Methodology
  12. The F-LLM Architecture
  13. Patch-wise Autoregressive Predictor
  14. Residual Estimator
  15. Closed-Loop Interaction
  16. ...and 23 more sections

Key Result

Proposition 4.1

Consider the linearized error dynamics $\Delta x_{t+1} \approx J_g \Delta x_t + \epsilon_t$. If the spectral radius $\rho(J_g) > 1$, the upper bound of the expected error norm grows exponentially with the horizon $H$.

Figures (8)

  • Figure 1: Comparison of standard autoregressive generation and the proposed stepwise correction strategy. (a) Standard autoregressive LLMs suffer from cumulative prediction drift, where minor deviations at each step are fed back as input, causing errors to accumulate over time. (b) The stepwise correction strategy intervenes in this process by adjusting the prediction at each step before it enters the next autoregressive loop, thereby preventing error propagation and stabilizing the trajectory.
  • Figure 2: The overall framework of F-LLM. The system operates as a closed-loop control mechanism where a frozen LLM as the Plant generates raw predictions, while a lightweight Residual Estimator as the Observer predicts trajectory deviations based on historical errors. The estimated residuals are injected back into the context via an in-loop correction mechanism to mitigate error accumulation.
  • Figure 3: Comparison of 96-step forecasts on ETTh1 and Weather. The frozen LLM produces locally reasonable predictions but gradually accumulates stepwise errors along the autoregressive rollout. In contrast, F-LLM compensates for structured residual bias and maintains trajectory alignment over long horizons.
  • Figure 4: Sensitivity analysis of F-LLM to the historical lookback length $T$ and patch length $L_p$ on the ETTh1 dataset. The results show that F-LLM maintains stable performance across a wide range of hyperparameter choices, indicating that the observed gains are not sensitive to specific context-length or tokenization settings.
  • Figure 5: Efficiency comparison between F-LLM and representative LLM4TS methods in terms of training and inference time as well as the number of tunable parameters, measured with the same batch size (224) on the ETTh1 dataset. F-LLM introduces only marginal computational overhead by adding a lightweight inference-time feedback module on top of frozen LLM backbones.
  • ...and 3 more figures

Theorems & Definitions (4)

  • Proposition 4.1: Exponential Error Growth
  • Theorem 4.1: Bounded Stability via Feedback
  • Proof C.1
  • Proof D.1