Adaptive parameters identification for nonlinear dynamics using deep permutation invariant networks

TL;DR

This work tackles the challenge of online parameter identification for nonlinear dynamical systems whose parameters evolve over time. It introduces two permutation-invariant set-encoding architectures, Deep Set and Set Transformer, to encode variable-length time-series data and predict dynamics parameters within a SINDy-based framework, enabling rapid online inference. The authors demonstrate the approach on Lotka-Volterra (local identification), Lorenz (global dynamics), and 1D heat-transfer problems (abnormality characterization), showing that Set Transformer provides robust extrapolation for longer sequences and that incorporating an ODE-consistency loss improves training. The results indicate meaningful improvements over the OASIS baseline and highlight practical potential for real-time forecasting and control in nonlinear systems, while outlining future work to handle PDEs, term activation/deactivation, and integration with physics-informed learning.

Abstract

The promising outcomes of dynamical system identification techniques, such as SINDy [Brunton et al. 2016], highlight their advantages in providing qualitative interpretability and extrapolation compared to non-interpretable deep neural networks [Rudin 2019]. These techniques suffer from parameter updating in real-time use cases, especially when the system parameters are likely to change during or between processes. Recently, the OASIS [Bhadriraju et al. 2020] framework introduced a data-driven technique to address the limitations of real-time dynamical system parameters updating, yielding interesting results. Nevertheless, we show in this work that superior performance can be achieved using more advanced model architectures. We present an innovative encoding approach, based mainly on the use of Set Encoding methods of sequence data, which give accurate adaptive model identification for complex dynamic systems, with variable input time series length. Two Set Encoding methods are used, the first is Deep Set [Zaheer et al. 2017], and the second is Set Transformer [Lee et al. 2019]. Comparing Set Transformer to OASIS framework on Lotka Volterra for real-time local dynamical system identification and time series forecasting, we find that the Set Transformer architecture is well adapted to learning relationships within data sets. We then compare the two Set Encoding methods based on the Lorenz system for online global dynamical system identification. Finally, we trained a Deep Set model to perform identification and characterization of abnormalities for 1D heat-transfer problem.

Figures (19)

• Figure 1: OASIS training process. $X_{jk}\in\mathbb{R}^{N^{'}\times l}$ and $U_{jk}\in\mathbb{R}^{N^{'}\times p}$ represent respectively state variables and control variables for the $j$-th set and the $k$-th sub domain, where $j \in [1, S]$ and $k \in [1, m]$. Here, $S$ is the number of sets and $m$ is the total number of sub domains. $l$, $p$, and $N^{'}$ represent respectively the number of state variables, control variables, and time windows.
• Figure 2: Deep neural network with two hidden layers with two inputs denoted as $X$ and $U$, and five outputs denoted as $\Xi$. The function $F$, which is trainable, depends on a set of variables denoted as $\Theta$ that governs the connections between these hidden layers.
• Figure 3: In top : two examples showcase the dynamics where the model can easily capture the local behavior with only one point in time (left image) and another one where the model will fail to capture the local behavior due to confusion (image at the right). In the bottom : example of dynamics with two state variables system (Lotka–Volterra) where the model could fail to capture the local dynamics at certain points.
• Figure 4: Set encoding overall architecture : The input sets, $X_{jk}$ and $U_{jk}$, represent respectively state variables and control variables for the $j$-th set and the $k$-th sub domain, that corresponds to the output coefficient $\Xi_{jk}$, $j \in [1, N]$ and $k \in [1, m]$. $N$ is the number of sets and $m$ is the total number of sub domains.
• Figure 5: Deep Set Architecture : the encoder is a DNN followed by a pooling transformation (max, mean, min ...), which is then decoded with another DNN.