Dynamical mean field approach to associative memory model with non-monotonic transfer functions

TL;DR

Dynamical mean-field theory is applied to the discrete-time synchronous retrieval dynamics of the non-monotonic model, which succeeds in accurately characterizing its macroscopic dynamical properties.

Abstract

The Hopfield associative memory model stores random patterns in synaptic couplings according to Hebb's rule and retrieves them through gradient descent on an energy function. This conventional setting, where neurons are assumed to have monotonic transfer functions, has been central to understanding associative memory. Morita (1993), however, showed that introducing non-monotonic transfer functions can dramatically enhance retrieval performance. While this phenomenon has been qualitatively examined, a full quantitative theory remains elusive due to the difficulty of analysis in the absence of an underlying energy function. In this work, we apply dynamical mean-field theory to the discrete-time synchronous retrieval dynamics of the non-monotonic model, which succeeds in accurately characterizing its macroscopic dynamical properties. We also derive conditions for retrieval states, and clarify their relation to previous studies. Our results provide new insights into the non-equilibrium retrieval dynamics of associative memory models.

Figures (9)

• Figure 1: (a): Non-monotonic transfer function of Eq. (6). (b): Piecewise-linear transfer function.
• Figure 2: Comparison of the readout overlap $M^t$ between (a) direct simulations and (b) DMFT results.
• Figure 3: Comparison of the output overlap $m^t$ between (a) direct simulations and (b) DMFT results.
• Figure 4: Heat map of the readout overlap $M^t$ at $t = 100$. The horizontal and vertical axes correspond to $\alpha$ and $M^0$, respectively. Vertical broken lines stand for the conventional storage capacity $\alpha_{\rm c} \simeq 0.138$.
• Figure 5: Time evolution of the feedback coefficients $\Lambda(t,s)$. (a,b): Successful retrieval. (c,d): Failed retrieval. Broken lines in (a) and (c) represent positions of data plotted in (b) and (d). To facilitate comparison, identical plot scales were used for both the successful and failed retrieval cases.