Adaptive Hopfield Network: Rethinking Similarities in Associative Memory

TL;DR

The paper introduces a probabilistic framework for correct retrieval in associative memory via a context-dependent variant distribution and replaces fixed similarity with a learnable adaptive similarity. Central to the approach is the similarity footprint, a multi-scale descriptor that enables effective approximation of the unknown $p_{\mathcal{V}}(\mathbf{x}|\boldsymbol\xi)$, which is then integrated into an adaptive Hopfield network (A-Hop). The authors prove optimal retrieval under noisy, masked, and biased variants and demonstrate state-of-the-art performance across memory retrieval, tabular classification, image classification, and multiple instance learning. Empirically, A-Hop shows robust retrieval under mixed corruptions and superior performance in downstream tasks, highlighting the practical impact of context-adaptive similarity for memory-based AI systems.

Abstract

Associative memory models are content-addressable memory systems fundamental to biological intelligence and are notable for their high interpretability. However, existing models evaluate the quality of retrieval based on proximity, which cannot guarantee that the retrieved pattern has the strongest association with the query, failing correctness. We reframe this problem by proposing that a query is a generative variant of a stored memory pattern, and define a variant distribution to model this subtle context-dependent generative process. Consequently, correct retrieval should return the memory pattern with the maximum a posteriori probability of being the query's origin. This perspective reveals that an ideal similarity measure should approximate the likelihood of each stored pattern generating the query in accordance with variant distribution, which is impossible for fixed and pre-defined similarities used by existing associative memories. To this end, we develop adaptive similarity, a novel mechanism that learns to approximate this insightful but unknown likelihood from samples drawn from context, aiming for correct retrieval. We theoretically prove that our proposed adaptive similarity achieves optimal correct retrieval under three canonical and widely applicable types of variants: noisy, masked, and biased. We integrate this mechanism into a novel adaptive Hopfield network (A-Hop), and empirical results show that it achieves state-of-the-art performance across diverse tasks, including memory retrieval, tabular classification, image classification, and multiple instance learning.

Figures (3)

• Figure 1: Visualization of probability density function $p_{{\mathcal{V}}}({\mathbf x} | {\boldsymbol\xi})$ for noisy, masked, biased, and noisy + masked variants. Darker regions indicate larger $p_{{\mathcal{V}}}({\mathbf x} | {\boldsymbol\xi})$, and the central dark point represents ${\boldsymbol\xi}$.
• Figure 2: Top: The architecture of adaptive Hopfield network (a), the training procedure for adaptive similarity (b) and its design choice (c). Bottom: An illustrative example of memory retrieval procedure involving two $16\times16$ image patterns and one query. The conventional proximitiy-based similarity (d) fails to retrieve the correct pattern while the adaptive similarity (e) succeeds.
• Figure 3: Retrieval accuracy ($\uparrow$) of 64-dimensional synthetic memory patterns.

Theorems & Definitions (27)

• definition 1
• definition 2
• definition 3
• definition 4
• definition 5
• definition 6
• definition 7
• theorem 1
• theorem 2
• definition 8