MODE: Learning compositional representations of complex systems with Mixtures Of Dynamical Experts

TL;DR

MODE introduces Mixture Of Dynamical Experts, a gating-based, sparse-regression framework for learning multiple governing laws from snapshot data. By combining either EM or neural gating with SINDy-like sparse dynamical terms, MODE discovers distinct dynamical regimes and transitions, enabling accurate unsupervised classification and long-horizon forecasting in noisy, regime-switching systems. The method is demonstrated on synthetic dynamics and real biological data, including scRNAseq-derived velocity and FUCCI-labeled cell cycle states, achieving strong clustering metrics and high-fidelity fate forecasting. This compositional approach provides interpretable governing equations for each regime and has practical implications for understanding branching and differentiation processes in biology.

Abstract

Dynamical systems in the life sciences are often composed of complex mixtures of overlapping behavioral regimes. Cellular subpopulations may shift from cycling to equilibrium dynamics or branch towards different developmental fates. The transitions between these regimes can appear noisy and irregular, posing a serious challenge to traditional, flow-based modeling techniques which assume locally smooth dynamics. To address this challenge, we propose MODE (Mixture Of Dynamical Experts), a graphical modeling framework whose neural gating mechanism decomposes complex dynamics into sparse, interpretable components, enabling both the unsupervised discovery of behavioral regimes and accurate long-term forecasting across regime transitions. Crucially, because agents in our framework can jump to different governing laws, MODE is especially tailored to the aforementioned noisy transitions. We evaluate our method on a battery of synthetic and real datasets from computational biology. First, we systematically benchmark MODE on an unsupervised classification task using synthetic dynamical snapshot data, including in noisy, few-sample settings. Next, we show how MODE succeeds on challenging forecasting tasks which simulate key cycling and branching processes in cell biology. Finally, we deploy our method on human, single-cell RNA sequencing data and show that it can not only distinguish proliferation from differentiation dynamics but also predict when cells will commit to their ultimate fate, a key outstanding challenge in computational biology.

Figures (10)

• Figure 1: Branching in NODE vs MODE.(Left) Ground truth progenitor cells in 2D advance along the blue trunk at a constant rate until they bifurcate into two fates, green and red, passing through an ambiguous switching region (inset). (Middle) Models that learn a single flow, like a NODE, can only reconcile switching zones by averaging. (Right) Instead, MODE learns a compositional flow with dedicated experts for trunk and branches, helping cells commit to their lineages.
• Figure 2: Plate diagram of MODE. Expert parameters $\Theta$ (with prior $p(\Theta)$) generate velocities $\dot{x}$ via $f_{\Theta_s}(x)$ under isotropic noise; the expert distribution optionally depends on $x$ (blue), in which case it gates states to specific experts, $s$.
• Figure 3: Elementary dynamics discovery on canonical systems. Each system challenges spatial clustering in distinct ways: (Bistable) spatially overlapping attractors with opposite rotational dynamics, (Lotka-Volterra) nonlinear predator-prey interactions creating complex phase structure, (Lorenz) chaotic flow with intricate spatial organization. Ground truth colors indicate true dynamical regimes. Clustering results colors indicate training data (gray) and estimated dynamical regimes (blue, orange). Accuracy of each method is displayed on the top-left of each subplots.
• Figure 4: MODE vs MLP baseline on synthetic biological switches.(Top row) Cells evolving according to the Goldbeter oscillator (first panel, blue) could escape ($p=0.15$ per time step) to a differentiated state (green X) when they pass through a region of low cyclin (gray box). The MLP (pictured, second panel) and SINDy try to use their single flow to explain the switching system, transiting near the differentiated branch's equilibrium before snaking back wildly towards the cycle. MODE (third panel) simply jettisons cells to the other expert with the appropriate probability since it has located the switching zone (argmax of $\pi(x)$, fourth panel). (Bottom). MODE's' performance advantage on simulated branching lineages (first panel) is similar: the MLP's (pictured, second panel) and SINDy's pushforward distribution blur the early split; MODE (third panel) better separates these branches since its experts locate the branches (fourth panel).
• Figure 5: MODE models cell cycle and differentiation dynamics.(a) Single cell RNA sequencing data from the U2OS bone cancer line was preprocessed for RNA velocity with scVelo (first three PCA dimensions shown). (b) We fit a 2-expert MODE to the first five PCA dimensions, revealing a sharp distinction between cycling and differentiating modes (red vs blue) averaged over ten random model initializations. (c) This closely matched ground truth scores computed from pseudotime, which allowed us to (d) accurately, and stably distinguish these two regimes. (e) Furthermore, MODE's stochastic rollout allows us to predict with substantial lead-in time (about a quarter cell cycle, see main text) when cells will commit to their differentiated fate outside of the cycle.