On the Identifiability of Regime-Switching Models with Multi-Lag Dependencies

TL;DR

This work develops a unified identifiability framework for regime-switching models with multi-lag dependencies, covering both Markov Switching Models (MSMs) and Switching Dynamical Systems (SDSs). For MSMs, the model is recast as a temporally structured finite mixture, yielding identifiability of the number of regimes and the multi-lag transitions under nonlinear Gaussian transitions, with nonparametric extensions. For SDSs, latent variables are identifiable up to permutation and scaling, enabling identifiability of regime-dependent latent causal graphs up to regime/node permutations. The theory operates in a fully unsupervised setting by constraining latent noise distributions and function classes and is complemented by a variational estimator aligned with the assumptions. Empirical results on synthetic data and real-world domains (neuroscience, finance, climate) demonstrate that identifiability leads to more trustworthy interpretability, enabling reliable regime segmentation and regime-dependent causal insights.

Abstract

Identifiability is central to the interpretability of deep latent variable models, ensuring parameterisations are uniquely determined by the data-generating distribution. However, it remains underexplored for deep regime-switching time series. We develop a general theoretical framework for multi-lag Regime-Switching Models (RSMs), encompassing Markov Switching Models (MSMs) and Switching Dynamical Systems (SDSs). For MSMs, we formulate the model as a temporally structured finite mixture and prove identifiability of both the number of regimes and the multi-lag transitions in a nonlinear-Gaussian setting. For SDSs, we establish identifiability of the latent variables up to permutation and scaling via temporal structure, which in turn yields conditions for identifiability of regime-dependent latent causal graphs (up to regime/node permutations). Our results hold in a fully unsupervised setting through architectural and noise assumptions that are directly enforceable via neural network design. We complement the theory with a flexible variational estimator that satisfies the assumptions and validate the results on synthetic benchmarks. Across real-world datasets from neuroscience, finance, and climate, identifiability leads to more trustworthy interpretability analysis, which is crucial for scientific discovery.

Figures (28)

• Figure 1: The generative models considered in this work. MSM (green) treats $\bm{z}_t$ directly as observations, while SDS (red) transforms $\bm{z}_t$ into observed $\bm{x}_t$.
• Figure 2: The red region shows our main result for parametric MSMs with Gaussian transitions (Sec. \ref{['sec:msm_theory']}). The green region indicates our more general identifiability result for nonparametric MSMs (App. \ref{['app:proof_identifiability']}). The blue region highlights that our proof technique extends to any switching model whose family lies within $\mathrm{conv}(\mathcal{P}^{T,M}_{\mathcal{A},\mathcal{B}})$.
• Figure 2: Per-season and global ENSO alignment with iMSM PCs. Peak correlation and lag with ENSO leading.
• Figure 3: Intuition behind Lemma \ref{['lemma:linear_independence_two_nonlinear_gaussians']}, where linear independence holds if, for any pair of functions (shown in green and purple), the intersection in the conditioned variable ($\bm{z}_{t-1}$) domain is zero-measured.
• Figure 4: Linear independence under finite mixtures of $\mathcal{P}^{T,M}_{\mathcal{A},\mathcal{B}}$ is proven by a nested induction strategy. The inner induction (blue box) constructs the product family $(\otimes^r\mathcal{P}^M_\mathcal{B},\ 1 \leq r \leq M)$, while the outer induction (red boxes) increases the length of the trajectory family.

Theorems & Definitions (35)

• Definition 1
• Definition 2
• Proposition 3
• Proposition 4
• Definition 5
• Definition 6
• Definition 7
• Theorem 8: Identifiability of parametric MSMs
• Remark 9
• Definition 10