On the Identifiability of Switching Dynamical Systems

TL;DR

The paper advances identifiability for sequential latent-variable models by first establishing identifiability for Markov Switching Models under nonlinear Gaussian transitions and then extending these results to Switching Dynamical Systems with nonlinear emissions. Identifiability is attained up to affine transformations and permutations, enabling neural-network parameterizations without requiring injective decoders. Estimation proceeds via EM for MSMs and variational inference for SDSs, with rigorous theoretical guarantees and practical algorithms. Empirically, identifiable MSMs/SDSs enable regime-dependent causal discovery in climate data and accurate segmentation of high-dimensional time series, including dancing videos, highlighting the method's interpretability and utility in complex sequential domains.

Abstract

The identifiability of latent variable models has received increasing attention due to its relevance in interpretability and out-of-distribution generalisation. In this work, we study the identifiability of Switching Dynamical Systems, taking an initial step toward extending identifiability analysis to sequential latent variable models. We first prove the identifiability of Markov Switching Models, which commonly serve as the prior distribution for the continuous latent variables in Switching Dynamical Systems. We present identification conditions for first-order Markov dependency structures, whose transition distribution is parametrised via non-linear Gaussians. We then establish the identifiability of the latent variables and non-linear mappings in Switching Dynamical Systems up to affine transformations, by leveraging identifiability analysis techniques from identifiable deep latent variable models. We finally develop estimation algorithms for identifiable Switching Dynamical Systems. Throughout empirical studies, we demonstrate the practicality of identifiable Switching Dynamical Systems for segmenting high-dimensional time series such as videos, and showcase the use of identifiable Markov Switching Models for regime-dependent causal discovery in climate data.

Figures (16)

• Figure 1: The generative model considered in this work, where the MSM is indicated in green and the SDS is indicated in red. The dashed arrows indicate additional dependencies which are accommodated by our theoretical results.
• Figure 2: Illustration of the intuition behind Lemma \ref{['lemma:linear_independence_two_nonlinear_gaussians']}, where linear independence holds if, for any pair of functions (shown in green and purple), the intersection in the domain of the conditioned variable ($\bm{z}_{t-1}$) is zero-measured.
• Figure 3: We assume $\bm{z}_t$ is transformed via $f$ with noise $\bm{\epsilon}_t$ at each time-step $t$ independently. We view this as a transformation on $\bm{z}_{1:T}$ via a factored $\mathcal{F}$ with noise $\mathcal{E}$.
• Figure 4: Synthetic experiment results on MSMs. (a) $L_2$ distance error using different transition functions with varying $T$. (b) $L_2$ distance error and (c) averaged $F_1$ score of non-linear data (cosine activations) with increasing states and dimensions.
• Figure 5: Reconstruction and segmentation (with ground truth) of a video generated from 2D latent variables sampled from a MSM (frame size $32\times 32$).

Theorems & Definitions (44)

• Definition 2.1
• Definition 2.2
• Definition 3.1
• Theorem 3.2
• Definition 3.3
• Definition 3.4
• Theorem 3.5
• Definition 1.1
• Definition 1.2
• Proposition 1.3