Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Data

TL;DR

This work introduces a hierarchical framework that enables to harvest group-level (multi-domain) information while retaining all single-domain characteristics, and demonstrates transfer learning and generalization to new parameter regimes, paving the way toward DSR foundation models.

Abstract

In science, we are often interested in obtaining a generative model of the underlying system dynamics from observed time series. While powerful methods for dynamical systems reconstruction (DSR) exist when data come from a single domain, how to best integrate data from multiple dynamical regimes and leverage it for generalization is still an open question. This becomes particularly important when individual time series are short, and group-level information may help to fill in for gaps in single-domain data. Here we introduce a hierarchical framework that enables to harvest group-level (multi-domain) information while retaining all single-domain characteristics, and showcase it on popular DSR benchmarks, as well as on neuroscience and medical data. In addition to faithful reconstruction of all individual dynamical regimes, our unsupervised methodology discovers common low-dimensional feature spaces in which datasets with similar dynamics cluster. The features spanning these spaces were further dynamically highly interpretable, surprisingly in often linear relation to control parameters that govern the dynamics of the underlying system. Finally, we illustrate transfer learning and generalization to new parameter regimes, paving the way toward DSR foundation models.

Figures (29)

• Figure 1: Illustration of the hierarchization framework.
• Figure 2: Example reconstructions from a hier-shPLRNN trained on short noisy observations ($T_{max}=1000$, $5\%$ observation noise) from the Lorenz-63, Rössler and Lorenz-96 systems for different $\{\rho,c,F\}$ (settings as in Sect. \ref{['sec:transfer_learning']} and \ref{['sec:interpretability']}). Shown trajectories were freely generated from a data-inferred initial state, using only subject-specific feature vectors $\bm{l}^{(j)}$ to determine the dynamical regime, and agree in their long-term temporal and geometrical structure with the ground truth (i.e., for times indefinitely beyond the short training sequences).
• Figure 3: a: Analysis of one-dimensional features for hierarchical models trained on observations from the Lorenz-63 (left) and Rössler (right) system for different values of $\rho$ and $c$. b: Explained variance ratio of the feature space PCs for a hier-shPLRNN trained on the Lorenz-63 with $N_{\text{feat}}=10$ vs. ratio for PCA directly on the parameters of individually trained models.
• Figure 4: PCA projection of the 6d feature space for a hier-shPLRNN trained on the Lorenz-63 with variation across all 3 ground truth parameters (each dot represents one parameter combination or 'subject'). From left to right, color-coding corresponds to parameters $\sigma$, $\rho$ and $\beta$, respectively. The learned feature space is highly structured and clearly distinguishes different dynamical regimes.
• Figure 5: a: Ground truth and simulated (freely generated) arterial flow velocity waves for $10$ representative subjects (selected by k-medoids, color-coded). b: $R^2$ scores for the $32$ ground truth haemodynamic parameters predicted from the learned feature vectors via linear regression. c: Ground truth and predicted values for the left ventricular ejection time (LVET) and pulse wave velocity (PWV). d: Variance explained by the first $5$ principal components of the learned feature space ($N_{\text{feat}}=12$) and the $32$d space spanned by the haemodynamic parameters.