Comparing Dynamical Models Through Diffeomorphic Vector Field Alignment

TL;DR

The paper tackles the problem of comparing learned dynamical systems when coordinate choices differ and motifs lie in low-dimensional manifolds within high-dimensional spaces. It introduces DFORM, a framework that learns a diffeomorphic coordinate transform to align vector fields via a pushforward-based orbital similarity loss, implemented with a Neural ODE-based flow atop a linear map. The authors validate linear and nonlinear alignment across synthetic linear and nonlinear systems, flows, and bifurcations, and demonstrate that DFORM can locate invariant manifolds and saddle limit cycles, as well as recover low-dimensional dynamics embedded in high-dimensional neural data like resting-state MINDy models. This approach offers a robust tool for mechanistic comparison, template-based motif discovery, and potential linearization insights for nonlinear dynamical systems in neuroscience and beyond.

Abstract

Dynamical systems models such as recurrent neural networks (RNNs) are increasingly popular in theoretical neuroscience for hypothesis-generation and data analysis. Evaluating the dynamics in such models is key to understanding their learned generative mechanisms. However, such evaluation is impeded by two major challenges: First, comparison of learned dynamics across models is difficult because there is no enforced equivalence of their coordinate systems. Second, identification of mechanistically important low-dimensional motifs (e.g., limit sets) is intractable in high-dimensional nonlinear models such as RNNs. Here, we propose a comprehensive framework to address these two issues, termed Diffeomorphic vector field alignment FOR learned Models (DFORM). DFORM learns a nonlinear coordinate transformation between the state spaces of two dynamical systems, which aligns their trajectories in a maximally one-to-one manner. In so doing, DFORM enables an assessment of whether two models exhibit topological equivalence, i.e., similar mechanisms despite differences in coordinate systems. A byproduct of this method is a means to locate dynamical motifs on low-dimensional manifolds embedded within higher-dimensional systems. We verified DFORM's ability to identify linear and nonlinear coordinate transformations using canonical topologically equivalent systems, RNNs, and systems related by nonlinear flows. DFORM was also shown to provide a quantification of similarity between topologically distinct systems. We then demonstrated that DFORM can locate important dynamical motifs including invariant manifolds and saddle limit sets within high-dimensional models. Finally, using a set of RNN models trained on human functional MRI (fMRI) recordings, we illustrated that DFORM can identify limit cycles from high-dimensional data-driven models, which agreed well with prior numerical analysis.

Figures (18)

• Figure 1: DFORM Schematic. A: Many efforts to compare learned models resort to assessment of limit sets, often by visualization. Here we plot the projection of simulated trajectories and numerically identified attractors of two models on the first three principal components (PCs) of their trajectories. B: We propose DFORM to learn a diffeomorphism that directly aligns vector fields. The mismatch between transformed and target vector fields defines the orbital similarity loss, which is used to optimize the diffeomorphism.
• Figure 2: Identification of transformations between equivalent linear systems. We randomly generated pairs of topologically equivalent linear systems of different sizes. Box plots showed the distribution of forward alignment between $\varphi_{*}f$ and $g$ after DFORM training, across 30 experiments for each of the four types of linear systems described in the main text (indicated by different colors). Bottom, middle and top horizontal lines in each plot showed 25, 50, and 75 percentiles respectively. Whiskers extend to the largest/smallest value within 1.5 times of interquartile range from the hinges. Dots represent outliers outside this range.
• Figure 3: Orbital similarity between linear systems with different signatures. A. Mean orbital similarity after DFORM alignment between linear systems. X and Y values indicate the signature of the first and second system under comparison, respectively. The signature was represented as an ordered pair, indicating the number of eigenvalues with negative and positive real parts. Color and number in each block indicate the mean orbital similarity across 20 experiments. B. Distribution of the similarity based on the concordance between the two systems' signatures. The five concordance levels are indicated by values along the horizontal axis and correspond to the fourth super/sub-diagonals to the main diagonal in panel A, respectively.
• Figure 4: Identification of transformations between nonlinear systems. We generated RNNs with 'low-rank plus random' connectivity and applied a random linear transformation (either orthogonal or non-orthogonal, as indicated by colors) to each system. Scatter plots showed the obtained Jacobian similarity (X axis) and fixed point similarity (Y axis) between each pair of DFORM-transformed and target systems. Experiments were grouped into panels by the size of the systems (16 to 128). Systems without nonzero stable equilibria were not shown in the figure as the fixed point similarity was meaningless, but their Jacobian similarity followed similar distribution as the ones shown.
• Figure 5: Identification of nonlinear transformation between Van der Pol oscillators. We generated two Van der Pol oscillator systems (see methods) with different bifurcation parameters $\mu = 0.2$ (left) and $\mu = 2$ (right). A linear DFORM model (middle left) and a nonlinear DFORM model (middle right) were trained using the same number of samples. Left and right panels: Dark contour plots visualized the sample distributions. Arrows showed the direction and magnitude of the vector fields (length normalized within each panel). Gray curves represented simulated trajectories. Middle panels: Pushforwards of the vector fields, distributions and trajectories in the left panel by the learned DFORM model. Arrows were colored according to their preimages in the left panels.

Theorems & Definitions (2)

• Theorem 1
• Theorem 2