Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Rank-Reduced Kalman Filter: Approximate Dynamical-Low-Rank Filtering In High Dimensions

Jonathan Schmidt, Philipp Hennig, Jörg Nick, Filip Tronarp

TL;DR

The paper introduces the Rank-Reduced Kalman Filter (RRKF), a deterministic, low-rank approach to Gaussian filtering and smoothing in high-dimensional state-space models. It combines dynamical low-rank approximation (DLRA) to solve Lyapunov equations for the predicted covariance with a low-rank, column-space–preserving update, and derives a backward smoothing kernel compatible with the low-rank representation. The method recovers the exact Kalman filter in the full-rank limit and achieves near-linear or quadratic scaling $O(n r^2 + m r^2 + r^3)$ under favorable assumptions, while remaining deterministic and competitive with ensemble methods in both mean and covariance accuracy. Empirical results across linear advection, London air-quality data, and large-scale spatio-temporal GP regression on rainfall demonstrate substantial accuracy gains over EnKF/ETKF with manageable computational costs, highlighting RRKF as a principled, scalable alternative for high-dimensional probabilistic state estimation.

Abstract

Inference and simulation in the context of high-dimensional dynamical systems remain computationally challenging problems. Some form of dimensionality reduction is required to make the problem tractable in general. In this paper, we propose a novel approximate Gaussian filtering and smoothing method which propagates low-rank approximations of the covariance matrices. This is accomplished by projecting the Lyapunov equations associated with the prediction step to a manifold of low-rank matrices, which are then solved by a recently developed, numerically stable, dynamical low-rank integrator. Meanwhile, the update steps are made tractable by noting that the covariance update only transforms the column space of the covariance matrix, which is low-rank by construction. The algorithm differentiates itself from existing ensemble-based approaches in that the low-rank approximations of the covariance matrices are deterministic, rather than stochastic. Crucially, this enables the method to reproduce the exact Kalman filter as the low-rank dimension approaches the true dimensionality of the problem. Our method reduces computational complexity from cubic (for the Kalman filter) to \emph{quadratic} in the state-space size in the worst-case, and can achieve \emph{linear} complexity if the state-space model satisfies certain criteria. Through a set of experiments in classical data-assimilation and spatio-temporal regression, we show that the proposed method consistently outperforms the ensemble-based methods in terms of error in the mean and covariance with respect to the exact Kalman filter. This comes at no additional cost in terms of asymptotic computational complexity.

The Rank-Reduced Kalman Filter: Approximate Dynamical-Low-Rank Filtering In High Dimensions

TL;DR

The paper introduces the Rank-Reduced Kalman Filter (RRKF), a deterministic, low-rank approach to Gaussian filtering and smoothing in high-dimensional state-space models. It combines dynamical low-rank approximation (DLRA) to solve Lyapunov equations for the predicted covariance with a low-rank, column-space–preserving update, and derives a backward smoothing kernel compatible with the low-rank representation. The method recovers the exact Kalman filter in the full-rank limit and achieves near-linear or quadratic scaling under favorable assumptions, while remaining deterministic and competitive with ensemble methods in both mean and covariance accuracy. Empirical results across linear advection, London air-quality data, and large-scale spatio-temporal GP regression on rainfall demonstrate substantial accuracy gains over EnKF/ETKF with manageable computational costs, highlighting RRKF as a principled, scalable alternative for high-dimensional probabilistic state estimation.

Abstract

Inference and simulation in the context of high-dimensional dynamical systems remain computationally challenging problems. Some form of dimensionality reduction is required to make the problem tractable in general. In this paper, we propose a novel approximate Gaussian filtering and smoothing method which propagates low-rank approximations of the covariance matrices. This is accomplished by projecting the Lyapunov equations associated with the prediction step to a manifold of low-rank matrices, which are then solved by a recently developed, numerically stable, dynamical low-rank integrator. Meanwhile, the update steps are made tractable by noting that the covariance update only transforms the column space of the covariance matrix, which is low-rank by construction. The algorithm differentiates itself from existing ensemble-based approaches in that the low-rank approximations of the covariance matrices are deterministic, rather than stochastic. Crucially, this enables the method to reproduce the exact Kalman filter as the low-rank dimension approaches the true dimensionality of the problem. Our method reduces computational complexity from cubic (for the Kalman filter) to \emph{quadratic} in the state-space size in the worst-case, and can achieve \emph{linear} complexity if the state-space model satisfies certain criteria. Through a set of experiments in classical data-assimilation and spatio-temporal regression, we show that the proposed method consistently outperforms the ensemble-based methods in terms of error in the mean and covariance with respect to the exact Kalman filter. This comes at no additional cost in terms of asymptotic computational complexity.
Paper Structure (42 sections, 6 theorems, 59 equations, 7 figures, 3 tables)

This paper contains 42 sections, 6 theorems, 59 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related work
  4. Background
  5. Square-root filtering
  6. Dynamical low-rank approximation
  7. Rank-reduced Kalman filtering
  8. Efficient inference in static models of low rank
  9. The rank-reduced filtering recursion
  10. The prediction equations
  11. The update equations
  12. Time complexity of the rank-reduced filtering recursion
  13. The rank-reduced smoothing recursion
  14. Approximating the backwards kernel
  15. Experiments
  16. ...and 27 more sections

Key Result

Proposition 1

Let $\mathrm{z}$ and $\mathrm{y}$ be two random variables governed by eq:reduced_static_model. Assume $r \leq m$ and consider the following singular value decomposition $(\mathrm{R}^{-1/2} \mathrm{C} \mathrm{\Pi}^{1/2} )^\ast = \mathrm{U} \mathrm{D} \mathrm{V}^\ast$, where $\mathrm{U}, \mathrm{D} \i Furthermore, let $\abs{\, \cdot \, }$ denote the matrix determinant. The marginal log-likelihood o

Figures (7)

  • Figure 1: Rank-$r$ approximations to the true KF covariance (left) for increasing $r$. The considered problem's true rank is $r^\ast = 7$ by construction, the state-dimension is $n = 1000$. The respective low-rank factors are shown above and left of their outer product. For $r \geq r^\ast$, the KF estimate is recovered. For $r > r^\ast$, the excess columns of the low-rank factor collapse (rightmost plot).
  • Figure 2: Performance of the low-rank filters on two different settings. In the truly low-rank linear advection problem (a), the RRKF achieves the optimal estimate () when $r$ exceeds the true rank of the problem. For the spatio-temporal air-quality regression (b), our method is consistently better.
  • Figure 3: How does the rate of spectral decay influence the low-rank approximation? The larger the spatial length scale of a spatio-temporal Matérn model, the faster the spectrum decays and the faster all methods converge to the KF estimate. The first and second row compare the mean estimates and the covariance estimates, respectively. The RRKF estimate is consistently closer and recovers the KF estimate () at $r = n$. The cumulative spectrum of the final-step KF covariance is shown in grey.
  • Figure 4: Best-case and worst-case asymptotic complexities of the low-rank filters. When all assumption:cheap-predictionassumption:diagonal-R are satisfied (A.), the elapsed time grows linearly with the state dimension $n$. If parts of the assumptions are not met (B.), the cost scales quadratically with the respective dimension.
  • Figure 5: Rainfall in Australia. A spatio-temporal GP regression problem with $n = 30\,911$ state dimensions, solved with the RRKF with $r = 1000$. The columns are three different days in the time series. The rows show the data, mean estimate, and Z-scores, respectively.
  • ...and 2 more figures

Theorems & Definitions (10)

  • Proposition 1
  • Corollary 1
  • Proposition 2
  • Corollary 2
  • Proposition 3
  • proof
  • proof
  • proof
  • Proposition B.1
  • Remark C.1