Back to the Continuous Attractor

TL;DR

The paper tackles the fragility of pure continuous attractors in neural systems by reframing analog memory through persistent slow manifolds. Using Fenichel’s persistence and fast-slow decomposition, it shows that small perturbations create invariant slow manifolds that maintain memory-like dynamics and can revive a continuous-attractor structure. Numerical experiments with task-trained RNNs reveal universal slow-manifold topologies near ring attractors across architectures and tasks, linking memory performance to the geometry of these manifolds. The findings argue that continuous attractors remain a valuable universal framework for understanding analog memory, even when exact attractors are unattainable in practice.

Abstract

Continuous attractors offer a unique class of solutions for storing continuous-valued variables in recurrent system states for indefinitely long time intervals. Unfortunately, continuous attractors suffer from severe structural instability in general--they are destroyed by most infinitesimal changes of the dynamical law that defines them. This fragility limits their utility especially in biological systems as their recurrent dynamics are subject to constant perturbations. We observe that the bifurcations from continuous attractors in theoretical neuroscience models display various structurally stable forms. Although their asymptotic behaviors to maintain memory are categorically distinct, their finite-time behaviors are similar. We build on the persistent manifold theory to explain the commonalities between bifurcations from and approximations of continuous attractors. Fast-slow decomposition analysis uncovers the persistent manifold that survives the seemingly destructive bifurcation. Moreover, recurrent neural networks trained on analog memory tasks display approximate continuous attractors with predicted slow manifold structures. Therefore, continuous attractors are functionally robust and remain useful as a universal analogy for understanding analog memory.

Figures (20)

• Figure 1: The critical weakness of continuous attractors is their inherent brittleness as they are rare in the parameter space, i.e., infinitesimal changes in parameters destroy the continuous attractor implemented in RNNs seung1996Renart2003. Some of the structure seems to remain; there is an invariant manifold that is topologically equivalent to the original continuous attractor. (A) Phase portraits for the bounded line attractor (Eq. \ref{['eq:TLN']}). Under perturbation of parameters, it bifurcates to systems without the continuous attractor. (B) The low-rank ring attractor approximation (Sec. \ref{['sec:supp:lowrank']}). Different topologies exist for different realizations of a low-rank attractor: different numbers of fixed points (4, 8, 12), or a limit cycle (right bottom). Yet, they all share the existence of a ring invariant set.
• Figure 2: Perturbations of different implementations and approximations of ring attractors lead to bifurcations that all leave the ring invariant manifold intact. For each model, the network dynamics is constrained to a ring manifold with stable fixed points (green) and saddle nodes (red). (A) Perturbations to noorman2024accurate. The ring attractor can be perturbed in systems with an even number of fixed points (FPs) up to $2N$ (stable and saddle points are paired). (B) Perturbations to a tanh approximation of a ring attractor seeholzer2017efficient. (C) Different Embedding Manifolds with Population-level Jacobians (EMPJ) approximations of a ring attractor pollock2020.
• Figure 3: Slow manifold approximation of different trained networks on the memory-guided saccade and angular velocity integration tasks. (A1) Output of an example trajectory on the angular velocity integration task. (A2) Output of example trajectories on the memory-guided saccade task. (B) An example fixed-point type solution to the memory-guided saccade task. Circles indicate fixed points of the system (filled for stable, empty for saddle) and the decoded angular value on the output ring is indicated with the color according to A1. (C) An example of a found solution to the angular velocity integration task. (D) An example slow-torus type solution to the memory-guided saccade task. The colored curves indicate stable limit cycles of the system. (E+F) The eigenvalue spectra for the trained networks in B and C show a gap between the first two largest eigenvalues.
• Figure 4: Temporal generalization validates theoretical predictions regardless of implementation detail. (A) Average accumulated angular error versus the maximum flow on the manifold (Eq. \ref{['eq:distance:ub']}), shown for finite time (task duration that the networks were trained on, $T_{1}$; filled markers) and at asymptotic time (hollow markers). (B) Normalized validation loss of all trained networks. (C) Average error and theoretical upper bound over time for two selected networks (corresponding to arrows in panel D). (D) Average asymptotic error is roughly inversely proportional to the number of fixed points. (E) Memory capacity is predictive of the average error.
• Figure S5: A slice of the parameter space of the BLA for a fixed $\epsilon_{11}$ and $\epsilon_{12}$.

Theorems & Definitions (24)

• Theorem 1: Persistent Manifold
• Proposition 1: Revival of continuous attractor
• Theorem 2
• proof
• Definition 1
• Definition 2
• Definition 3
• Theorem 3: Theorem 1 in Jones1995
• Theorem 4: One Dimensional Equivalence
• Theorem 5: $\epsilon$-Neighborhood Theoremguillemin2010differential