Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Finding separatrices of dynamical flows with Deep Koopman Eigenfunctions

Kabir V. Dabholkar, Omri Barak

TL;DR

The paper tackles the problem of identifying separatrices in high-dimensional multistable dynamical systems by learning scalar Koopman eigenfunctions with positive eigenvalues, which vanish on the separatrix. The authors propose a deep-network-based KEF that satisfies the Koopman PDE \nabla \psi(\boldsymbol{x}) \cdot f(\boldsymbol{x}) = \lambda \psi(\boldsymbol{x}) with \lambda>0, trained via a PDE residual and a ratio loss that avoids trivial solutions, plus a balance regularizer to mitigate degeneracy. They demonstrate robust separatrix recovery across synthetic models, ecological networks, and large neural network dynamics, and show how the KEFs enable gradient-based, minimal-norm perturbations to cross basin boundaries, with applications to neuroscience interventions (e.g., optogenetics). The work provides a scalable, data-driven framework that complements local fixed-point analyses and offers practical tools for predicting and steering transitions in complex dynamical landscapes. The method's high-dimensional applicability, together with its ability to design targeted perturbations, has potential impact in neuroscience, ecology, and engineered systems where boundary-crossing transitions are critical.

Abstract

Many natural systems, including neural circuits involved in decision making, are modeled as high-dimensional dynamical systems with multiple stable states. While existing analytical tools primarily describe behavior near stable equilibria, characterizing separatrices--the manifolds that delineate boundaries between different basins of attraction--remains challenging, particularly in high-dimensional settings. Here, we introduce a numerical framework leveraging Koopman Theory combined with Deep Neural Networks to effectively characterize separatrices. Specifically, we approximate Koopman Eigenfunctions (KEFs) associated with real positive eigenvalues, which vanish precisely at the separatrices. Utilizing these scalar KEFs, optimization methods efficiently locate separatrices even in complex systems. We demonstrate our approach on synthetic benchmarks, ecological network models, and high-dimensional recurrent neural networks trained on either neuroscience-inspired tasks or fit to real neural data. Moreover, we illustrate the practical utility of our method by designing optimal perturbations that can shift systems across separatrices, enabling predictions relevant to optogenetic stimulation experiments in neuroscience.

Finding separatrices of dynamical flows with Deep Koopman Eigenfunctions

TL;DR

The paper tackles the problem of identifying separatrices in high-dimensional multistable dynamical systems by learning scalar Koopman eigenfunctions with positive eigenvalues, which vanish on the separatrix. The authors propose a deep-network-based KEF that satisfies the Koopman PDE \nabla \psi(\boldsymbol{x}) \cdot f(\boldsymbol{x}) = \lambda \psi(\boldsymbol{x}) with \lambda>0, trained via a PDE residual and a ratio loss that avoids trivial solutions, plus a balance regularizer to mitigate degeneracy. They demonstrate robust separatrix recovery across synthetic models, ecological networks, and large neural network dynamics, and show how the KEFs enable gradient-based, minimal-norm perturbations to cross basin boundaries, with applications to neuroscience interventions (e.g., optogenetics). The work provides a scalable, data-driven framework that complements local fixed-point analyses and offers practical tools for predicting and steering transitions in complex dynamical landscapes. The method's high-dimensional applicability, together with its ability to design targeted perturbations, has potential impact in neuroscience, ecology, and engineered systems where boundary-crossing transitions are critical.

Abstract

Many natural systems, including neural circuits involved in decision making, are modeled as high-dimensional dynamical systems with multiple stable states. While existing analytical tools primarily describe behavior near stable equilibria, characterizing separatrices--the manifolds that delineate boundaries between different basins of attraction--remains challenging, particularly in high-dimensional settings. Here, we introduce a numerical framework leveraging Koopman Theory combined with Deep Neural Networks to effectively characterize separatrices. Specifically, we approximate Koopman Eigenfunctions (KEFs) associated with real positive eigenvalues, which vanish precisely at the separatrices. Utilizing these scalar KEFs, optimization methods efficiently locate separatrices even in complex systems. We demonstrate our approach on synthetic benchmarks, ecological network models, and high-dimensional recurrent neural networks trained on either neuroscience-inspired tasks or fit to real neural data. Moreover, we illustrate the practical utility of our method by designing optimal perturbations that can shift systems across separatrices, enabling predictions relevant to optogenetic stimulation experiments in neuroscience.

Paper Structure

This paper contains 28 sections, 1 theorem, 55 equations, 10 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Results
  4. KEFs as Scalar Separatrix Indicators
  5. Challenges and Solutions
  6. Degeneracy across basins.
  7. Demonstrations
  8. Discussion
  9. Analytical KEF derivation in 1D bistable system
  10. Eigenfunction Degeneracy in higher dimensions
  11. Relation of our definition to the Koopman Operator
  12. KEF degeneracy in randomly initialised DNN solutions
  13. Curve-based validation approach
  14. Neural network architectures
  15. Optimisation
  16. ...and 13 more sections

Key Result

Proposition 1

Let $\psi:\mathcal{X} \setminus \bigl(\cup_k A_k\bigr)\to\mathbb R$ be a continuous Koopman eigenfunction with positive eigenvalue $\lambda>0$: and have constant sign near all the attractors $A_k$. Suppose $\boldsymbol{x} \in \mathcal{X} \setminus \bigl(\cup_k A_k\bigr)$ satisfies $\psi(\boldsymbol{x})=0$ and, for every $\varepsilon>0$, the ball $B_d(\boldsymbol{x},\varepsilon)$ contains points $

Figures (10)

  • Figure 1: (A) Phase-portrait of a 2D bistable system. The two attractors can signify different choices, and therefore the direction between them is called the choice direction. External input pushes the system across the separatrix letting it relax to the other attractor. The optimal perturbation has a different direction. (B) The kinetic energy vanishes at the fixed points (green 'o' stable and '+' unstable) but does not reveal the full separatrix. (C) We aim to learn a scalar function $\psi(x)$ that vanishes precisely on the separatrix.
  • Figure 2: Mapping the high dimensional dynamics of $\boldsymbol x$ with an unstable manifold -- the separatrix -- to the one dimensional linear dynamics of $\psi$ with instability at $\psi=0$, i.e., $\lambda>0$. We approximate the mapping $\psi(\boldsymbol x)$ with a DNN.
  • Figure 3: Our method to approximate KEFs in three bistable systems. (A) A 1D system $\dot x=x-x^3$. The curve shows $f(x)$, and its fixed points in light green -- 'o's stable and 'x's unstable. (B) The kinetic energy $q(x)$ of this system. (C) the true KEF \ref{['eq:1DbistableKEF']} and its DNN approximation obtained by our method. (D,E,F) Damped Duffing Oscillator in 2D (G,H,I) 2-unit GRU chung_empirical_2014 RNN trained on 1-bit flip flop (1BFF) sussillo_opening_2013 and our KEFs.
  • Figure 4: Top: In the presence of multiple basins, a KEF can collapse to zero within a single basin. This degeneracy is realised by multiplying the KEF with a piecewise constant KEF with $\lambda=0$ and invoking \ref{['eq:KEFproductrule']}. This example corresponds to $\dot x=x-x^3$. We introduce a regularisation term \ref{['eq:balanceloss']} to encourage the mean value $\langle \psi \rangle\approx0$. This encourages solutions with sign changes across basins. Bottom: In higher dimensions, degeneracy arises from directional ambiguity in solutions. We visualise the analytical solution \ref{['eq:2Ddegeneracy']} for $\dot x = x-x^3;\dot y = y-y^3$. We address this by sampling from multiple local distributions around separatrix points and training an ensemble of KEFs.
  • Figure 5: Two-bit flip flop task in a 3-unit GRU. The system has 4 stable fixed points (light-green points). (A,B) Two KEFs obtained by our method. They complement each other as they each captures a separatrix along one direction. (C) Use of KEF to design minimal perturbations that push trajectories across the separatrix.
  • ...and 5 more figures

Theorems & Definitions (2)

  • Proposition 1: Sign change at a zero $\Rightarrow$ separatrix point
  • proof