Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Model-free Online Learning for the Kalman Filter: Forgetting Factor and Logarithmic Regret

Jiachen Qian, Yang Zheng

TL;DR

This paper tackles online prediction for unknown, non-explosive linear stochastic systems by proposing Online Prediction with Forgetting (OPF), a model-free algorithm that balances the regression model through exponential forgetting. The authors show that re-weighting alone is insufficient for the imbalanced Kalman-filter regression and demonstrate that forgetting induces a generalized ridge regression inductive bias, enabling a stable, balanced learning process. A central contribution is a sharpened regret bound of $O(\log^{3} N)$ with high probability, achieved via Hanson-Wright martingale analysis and accumulation-error cancellation across epoch-based horizons. The results yield a practical, provably robust approach to model-free Kalman-filter-like prediction with meaningful performance gains over prior online-prediction methods, especially in marginally stable regimes.

Abstract

We consider the problem of online prediction for an unknown, non-explosive linear stochastic system. With a known system model, the optimal predictor is the celebrated Kalman filter. In the case of unknown systems, existing approaches based on recursive least squares and its variants may suffer from degraded performance due to the highly imbalanced nature of the regression model. This imbalance can easily lead to overfitting and thus degrade prediction accuracy. We tackle this problem by injecting an inductive bias into the regression model via {exponential forgetting}. While exponential forgetting is a common wisdom in online learning, it is typically used for re-weighting data. In contrast, our approach focuses on balancing the regression model. This achieves a better trade-off between {regression} and {regularization errors}, and simultaneously reduces the {accumulation error}. With new proof techniques, we also provide a sharper logarithmic regret bound of $O(\log^3 N)$, where $N$ is the number of observations.

Model-free Online Learning for the Kalman Filter: Forgetting Factor and Logarithmic Regret

TL;DR

This paper tackles online prediction for unknown, non-explosive linear stochastic systems by proposing Online Prediction with Forgetting (OPF), a model-free algorithm that balances the regression model through exponential forgetting. The authors show that re-weighting alone is insufficient for the imbalanced Kalman-filter regression and demonstrate that forgetting induces a generalized ridge regression inductive bias, enabling a stable, balanced learning process. A central contribution is a sharpened regret bound of with high probability, achieved via Hanson-Wright martingale analysis and accumulation-error cancellation across epoch-based horizons. The results yield a practical, provably robust approach to model-free Kalman-filter-like prediction with meaningful performance gains over prior online-prediction methods, especially in marginally stable regimes.

Abstract

We consider the problem of online prediction for an unknown, non-explosive linear stochastic system. With a known system model, the optimal predictor is the celebrated Kalman filter. In the case of unknown systems, existing approaches based on recursive least squares and its variants may suffer from degraded performance due to the highly imbalanced nature of the regression model. This imbalance can easily lead to overfitting and thus degrade prediction accuracy. We tackle this problem by injecting an inductive bias into the regression model via {exponential forgetting}. While exponential forgetting is a common wisdom in online learning, it is typically used for re-weighting data. In contrast, our approach focuses on balancing the regression model. This achieves a better trade-off between {regression} and {regularization errors}, and simultaneously reduces the {accumulation error}. With new proof techniques, we also provide a sharper logarithmic regret bound of , where is the number of observations.
Paper Structure (42 sections, 6 theorems, 152 equations, 7 figures, 2 tables, 1 algorithm)

This paper contains 42 sections, 6 theorems, 152 equations, 7 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries and Problem Formulation
  3. Background on the Kalman Filter
  4. Model-free Prediction and Regret
  5. Long-term Memory and Forgetting
  6. Online Prediction with Forgetting
  7. Online Balanced Regression via Forgetting
  8. Inductive Bias from Forgetting
  9. Logarithmic Regret of the OPF Algorithm
  10. A Sharper Logarithmic Regret
  11. Proof Sketches for the Shaper Regret
  12. Numerical Experiments
  13. Conclusion
  14. Linear Stochastic System
  15. Basic statistical properties of random variables
  16. ...and 27 more sections

Key Result

Theorem 1

Consider the linear stochastic system linearSystem with $Q \succ 0, R \succ 0$, and suppose aspOb-main-textaspDiagonal hold. Choose the forgetting factor $\gamma \in (\rho(A-LC),1]$, and the parameters in algPrediction as where $\kappa$ represents the order of the largest Jordan block of $A$ corresponding to the eigenvalue $1$. Fix a horizon $N > T_{\mathrm{init}}$ and a failure probability $\del

Figures (7)

  • Figure 1: Illustration of the overfitting effect in a simple example. The black curve shows the ground truth that has a highly imbalanced structure, and the blue curve is the estimated model using method provided in tsiamis9894660 that overfits the small blocks.
  • Figure 2: Performance comparison of online prediction with different forgetting strategies: (a) our proposed forgetting; (b) traditional forgetting in jacobsen2024online.
  • Figure 3: Effect of our forgetting factor on different parts of regret $\mathcal{R}_N$ in \ref{['wholeRegret']}.
  • Figure 4: Illustration figure for the comparison of regret for different methods.
  • Figure 5: Illustration figure for the problem of numerical stability
  • ...and 2 more figures

Theorems & Definitions (12)

  • Remark 1: Forgetting strategies
  • Theorem 1
  • Remark 2
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Lemma 3
  • proof
  • Lemma 4
  • ...and 2 more