Dynamic Manifold Hopfield Networks for Context-Dependent Associative Memory

TL;DR

Dynamic reorganization of attractor manifold geometry is established as a principled mechanism for context-dependent remapping in neural associative memory in associative retrieval.

Abstract

Neural population activity in cortical and hippocampal circuits can be flexibly reorganized by context, suggesting that cognition relies on dynamic manifolds rather than static representations. However, how such dynamic organization can be realized mechanistically within a unified dynamical system remains unclear. Continuous Hopfield networks provide a classical attractor framework in which neural dynamics follow gradient descent on a fixed energy landscape, constraining retrieval within a static attractor manifold geometry. Extending this approach, we introduce Dynamic Manifold Hopfield Networks (DMHN), continuous dynamical models in which contextual modulation dynamically reshapes attractor geometry, transforming a static attractor manifold into a context-dependent family of neural manifolds. In DMHN, network interactions are learned in a data-driven manner, to intrinsically deform the geometry of its attractor manifold across cues without explicit context-specific parameterization. As a result, in associative retrieval, DMHN achieve substantially higher capacity and robustness than classical and modern Hopfield networks: when storing $2N$ patterns in a network of $N$ neurons, DMHN attain reliable retrieval with an average accuracy of 64%, compared with 1% and 13% for classical and modern variants, respectively. Together, these results establish dynamic reorganization of attractor manifold geometry as a principled mechanism for context-dependent remapping in neural associative memory.

Figures (4)

• Figure 1: Overview of Dynamic Manifold Hopfield Networks (DMHN). (a) Experiences across contexts give rise to distinct neural manifolds perich2025neural, suggesting that contextual changes reshape the underlying energy landscape and modulate attractor geometry. (b) Associative retrieval: a corrupted cue evolves through continuous attractor dynamics to produce a completed output, where the cue serves as the contextual signal. (c) Hebbian learning yields fixed attractor structures, whereas BP-based learning uses cue-target pairs to enable cue-conditioned retrieval. (d) A two-dimensional retrieval example illustrating challenging cue-attractor associations. (e,f) Comparison of CHN, S-DMHN, and DMHN in dynamics and learned basin geometry. (g) Corresponding energy landscapes: CHN and S-DMHN exhibit fixed landscapes, whereas DMHN induce cue-dependent effective landscapes. (h) Decomposition of DMHN energy into cue-independent (static) and cue-dependent (dynamic) terms, whose interaction generates dynamic manifold reorganization.
• Figure 2: Cue-dependent dynamic manifolds in DMHN. (a) Network state trajectories under continuously varying cues. As the cue changes, population activity remains constrained to a low-dimensional manifold whose geometry is dynamically reshaped, demonstrating that retrieval dynamics are governed by cue-dependent manifold deformation rather than transitions among fixed attractors. (b) A more complex two-dimensional retrieval task, in which 6 distinct cues converge to 4 target attractors. Unlike (a), cues are corrupted and held fixed during retrieval, and network dynamics are initialized from the noisy cue at onset.
• Figure 3: Associative memory retrieval framework and dynamics. (a) Associative retrieval settings. Binary and continuous-valued patterns are corrupted to generate cues, which are iteratively evolved toward target memories. Corruption includes bit flips, masking, or additive Gaussian noise. A retrieval is considered correct when the final error falls below a predefined threshold. (b) Training procedure. For binary retrieval, corrupted cues are iteratively evolved and a retrieval loss between the retrieved state and the target pattern is used to train the Hopfield network. For continuous-valued retrieval, target and cue embeddings are evolved in a shared latent space. Retrieval loss is applied between the retrieved embedding and the target embedding, while a reconstruction loss trains the encoder-decoder to reconstruct both targets and cues. (c,d) Retrieval dynamics under different memory loads. Representative retrieval trajectories for CHN, MHN, and DMHN when storing $0.2N$ patterns (c) and $2N$ patterns (d). For each dataset, a corrupted cue initializes the network, and successive states are shown across retrieval iterations. Under higher memory load, CHN and MHN exhibit increased distortion and convergence to mixed or spurious states, whereas DMHN trajectories remain more aligned with the target patterns. This trend is observed for both binary datasets (benchmark, imbalanced, and MNIST) and continuous-valued image representations (CIFAR10). Red boxes indicate retrievals considered correct according to the error threshold predefined in (a).
• Figure 4: Pattern retrieval performance and robustness across datasets. (a) Retrieval accuracy as a function of stored patterns for CHN, MHN, and DMHN. Right: zoomed view near the classical capacity limit. (b) Overlap distributions between retrieved states and target patterns across memory loads. Dashed lines denote the correct-retrieval threshold and the baseline overlap of the initial state. (c--e) Retrieval performance under heterogeneous binary patterns (imbalanced, MNIST) and continuous-valued images (CIFAR10). (f) Ablation results across datasets. (g) Attractor activity statistics learned under CIFAR10 evaluation.