dynoGP: Deep Gaussian Processes for dynamic system identification

TL;DR

The paper tackles probabilistic dynamic system identification by introducing dynoGP, a hierarchical deep Gaussian Process that stacks dynamic GPs (modeling stochastic LTI dynamics) with static GPs (modeling nonlinearities). It derives closed-form mean and covariance expressions for diagonal state matrices, enabling efficient inference via stochastic variational methods and inducing points, and demonstrates the approach on Wiener, Wiener-Hammerstein, and real-world benchmarks with uncertainty quantification. Key contributions include the analytic treatment of dynamic GP layers, the Wiener/LIN-NONLIN-LIN architectures, and a scalable variational framework implemented in GPyTorch. The results show dynoGP achieves competitive or superior predictive performance and provides principled uncertainty estimates, making it valuable for reliable decision-making in control and forecasting tasks.

Abstract

In this work, we present a novel approach to system identification for dynamical systems, based on a specific class of Deep Gaussian Processes (Deep GPs). These models are constructed by interconnecting linear dynamic GPs (equivalent to stochastic linear time-invariant dynamical systems) and static GPs (to model static nonlinearities). Our approach combines the strengths of data-driven methods, such as those based on neural network architectures, with the ability to output a probability distribution. This offers a more comprehensive framework for system identification that includes uncertainty quantification. Using both simulated and real-world data, we demonstrate the effectiveness of the proposed approach.

Figures (6)

• Figure 1: Example: system identification of an LTI system using Gaussian Processes. The top plot shows the training data, while the bottom plot reports the predictions along with the 95% credible interval in blue. The values to the left of the vertical bar represent the last 1000 values of the training set, while those to the right correspond to the test set.
• Figure 2: Example: system identification with dynoGP. Top: training data (output signal). Center: dynoGP with only a dynamic layer; values on the left of the vertical bar represent the last 1000 values of the training set, while those on the right correspond to the test set. Bottom: dynoGP with Wiener architecture. The last two plots show the posterior mean (blue line), 95% credible interval (darker blue-region), 99.7% credible interval (lighter blue-region).
• Figure 3: Predictive MAE and CRPS for $15 \leq t \leq 25$ for dynoNet, dynoGP and GP-NARX for simulated data from a Wiener system.
• Figure 4: Wiener-Hammerstein model with process noise. The output signal is in orange, the predictive posterior mean and 95% credible interval for dynoGP are shown for all testing data (top) and for a subset of testing data (bottom).
• Figure 5: CED: The first plot shows the input signal. Values on the left of the vertical bar represent the 400 values of the training set, while those on the right correspond to the test set. The second plot shows the values of the output signal (in orange) and the predictive posterior mean and 95% credible interval for dynoGP in blue.

Theorems & Definitions (10)

• Example 1
• Remark 1
• Lemma 1
• Proposition 1
• Proposition 2
• Proposition 3
• Example 2
• Remark 2
• Example 3
• Remark 3