On the importance of structural identifiability for machine learning with partially observed dynamical systems

TL;DR

This work addresses the challenge of structurally unidentifiable parameters in dynamical models used for time series classification. It introduces Structural Identifiability Mapping (SIM), which maps MAP estimates from unidentifiable models to identifiable parameter combinations, enabling classifiers to operate in a space where outputs are uniquely determined. Through four biomedical-inspired dynamical systems, SIM consistently improves generalisation, especially when training data are limited or noisy, and remains effective across dense, sparse, and irregular time grids. The approach preserves interpretability by allowing insights to be translated back to the original parameter space, and it highlights the importance of incorporating structural identifiability analysis into ML workflows that rely on mechanistic dynamics.

Abstract

The successful application of modern machine learning for time series classification is often hampered by limitations in quality and quantity of available training data. To overcome these limitations, available domain expert knowledge in the form of parametrised mechanistic dynamical models can be used whenever it is available and time series observations may be represented as an element from a given class of parametrised dynamical models. This makes the learning process interpretable and allows the modeller to deal with sparsely and irregularly sampled data in a natural way. However, the internal processes of a dynamical model are often only partially observed. This can lead to ambiguity regarding which particular model realization best explains a given time series observation. This problem is well-known in the literature, and a dynamical model with this issue is referred to as structurally unidentifiable. Training a classifier that incorporates knowledge about a structurally unidentifiable dynamical model can negatively influence classification performance. To address this issue, we employ structural identifiability analysis to explicitly relate parameter configurations that are associated with identical system outputs. Using the derived relations in classifier training, we demonstrate that this method significantly improves the classifier's ability to generalize to unseen data on a number of example models from the biomedical domain. This effect is especially pronounced when the number of training instances is limited. Our results demonstrate the importance of accounting for structural identifiability, a topic that has received relatively little attention from the machine learning community.

Figures (18)

• Figure 1: Geometric intuition behind the mechanism of SIM with data from the toy model. Panel a) depicts a binary classification problem which illustrates how training data can be oriented along manifolds of the form $\Phi = g(a,b) = a b$. Panel b) shows the representation of the same data after applying SIM. The decision boundary between the two classes becomes simpler and, in this special case, the data even become linearly separable in $\Phi$-space.
• Figure 2: Catenary $n$-compartmental model.
• Figure 3: Compartment Model with a Loop (CML).
• Figure 4: Binary classification task for time series from the batch reactor model. Displayed are 10 time series per class. Observational noise simulated is normally distributed with standard deviation $\sigma = 3$.
• Figure 5: Experiment 1 showing improved classification with partially observed batch reactor model due to SIM. Displayed are learning curves obtained from classifier training based on the fully observed (FO) dynamical model (dotted green) and the partially observed (PO) dynamical model, with and without application of SIM (marked with solid blue and dashed orange curves, respectively). The training and test data used were generated on the dense time grid $t_{\text{dense}}$ with fixed observational noise $\sigma = 1$ on each observed component.