Efficient Transformed Gaussian Process State-Space Models for Non-Stationary High-Dimensional Dynamical Systems

TL;DR

GPSSMs struggle with high-dimensional, non-stationary dynamics due to the need for multiple independent stationary GPs, leading to high computational and parametric costs. The authors propose ETGPSSM, which replaces per-dimension GPs with a single shared GP whose outputs are warped by input-dependent normalizing flows, making the transition dynamics non-stationary while maintaining scalability. They derive a flexible variational ELBO that leverages EnKF for latent-state inference, enabling end-to-end optimization without explicit state posteriors. Comprehensive experiments across non-stationary learning, chaotic high-dimensional filtering, and real-world forecasting demonstrate strong accuracy and substantial efficiency gains, validating the approach for scalable probabilistic modeling of complex, high-dimensional dynamical systems.

Abstract

Gaussian process state-space models (GPSSMs) offer a principled framework for learning and inference in nonlinear dynamical systems with uncertainty quantification. However, existing GPSSMs are limited by the use of multiple independent stationary Gaussian processes (GPs), leading to prohibitive computational and parametric complexity in high-dimensional settings and restricted modeling capacity for non-stationary dynamics. To address these challenges, we propose an efficient transformed Gaussian process state-space model (ETGPSSM) for scalable and flexible modeling of high-dimensional, non-stationary dynamical systems. Specifically, our ETGPSSM integrates a single shared GP with input-dependent normalizing flows, yielding an expressive non-stationary implicit process prior that can capture complex transition dynamics while significantly reducing model complexity. For the inference of the implicit process, we develop a variational inference algorithm that jointly approximates the posterior over the underlying GP and the neural network parameters defining the normalizing flows. To avoid explicit variational parameterization of the latent states, we further incorporate the ensemble Kalman filter (EnKF) into the variational framework, enabling accurate and efficient state estimation. Extensive empirical evaluations on synthetic and real-world datasets demonstrate the superior performance of our ETGPSSM in system dynamics learning, high-dimensional state estimation, and time-series forecasting, outperforming existing GPSSMs and neural network-based SSMs in terms of computational efficiency and accuracy.

Figures (6)

• Figure 1: Graphical representation of GPSSM. The white circles represent the latent variables, while the gray circles represent the observable variables. The thick horizontal bar represents a set of fully connected, mutually correlated nodes, specifically, the GP.
• Figure 2: Two-dimensional regression example comparing GP models. The non-stationary ETGP accurately captures local variations, outperforming stationary warped GPs and independent GPs.
• Figure 3: Non-stationary ETGP transition function in SSMs
• Figure 4: Non-stationary kink transition function learning performance (mean $\pm$$2 \sigma$) using various methods across different levels of emission noise ($\sigma_{\mathrm{R}}^2 \in \{0.0008, 0.008, 0.08, 0.8\}$, from top to bottom). The blue curve (---) represents the learned mean function, while the red line (---) indicates the true system transition function. The shaded region depicts the total predictive uncertainty for the state transition, which includes the estimated transition noise level $(\pm 2 \, \hat{\sigma}_Q)$ for all methods, plus function uncertainty for probabilistic models (BNN/GP).
• Figure 5: (Left) Comparison of the total number of parameters (including variational and model parameters). In ETGPSSM, the count is dominated by the GP ($M d_x + 3 + M + M^2$) and the neural network with architecture $d_x \!\to\! 128 \!\to\! 64 \!\to\! 2d_x$ ($258\,d_x + 8384$), resulting in overall linear scaling in $d_x$. In GPSSM, the parameters are dominated by the $d_x$ independent GPs, ($M d_x^2 + (3 + M + M^2) d_x$), leading to quadratic scaling in $d_x$. (Right) ETGPSSM maintains low computational costs as $d_x$ increases, while GPSSM exhibits linear growth.

Theorems & Definitions (20)

• Definition 1
• Definition 2
• Example 1
• Example 2
• Example 3
• Example 4
• Corollary 1
• proof
• Remark 1
• Proposition 1