Testing the Role of Diagonal Interactions in High-Order Hopfield Models via Dynamical Mean-Field Theory

Abstract

High-order extensions of the Hopfield model are known to exhibit dramatically enhanced storage capacity at equilibrium, while their dynamical retrieval properties remain less well understood. In our previous work, we carried out a dynamical mean-field theory (DMFT) analysis of the Krotov--Hopfield-type dense associative memory and found that the transition between successful and failed retrieval is accompanied by pronounced slow dynamics. As a consequence, the effective basin of attraction observed in numerical simulations extends well beyond that predicted by equilibrium statistical mechanics. A natural hypothesis is that this discrepancy originates from diagonal (self-interaction) contributions in the Krotov--Hopfield model, which generate a large number of lower-order interaction terms and may induce glassy relaxation near the retrieval boundary. To test this hypothesis, we analyze an alternative high-order associative memory model, namely the Abbott--Arian-type $p$-body Hopfield model, in which such diagonal contributions are absent by construction. Using dynamical mean-field theory, we derive an effective single-site process together with closed macroscopic equations governing the retrieval dynamics. Our analysis reveals that both slow dynamics and a substantial enlargement of the apparent basin of attraction persist even in this model. These results indicate that the dynamical slowdown near the retrieval boundary cannot be attributed primarily to diagonal self-interaction effects, but instead originates from intrinsic properties of high-order interactions.

Figures (4)

• Figure 1: Time evolution of the overlap $m(t)$ for $p=3$ and loading level $\alpha=0.05,0.1,0.15,0.2$. Curves correspond to different initial overlaps $m(0)$. Circles: DMFT prediction, obtained by solving the effective single-site equations with $N_{\rm s} = 10^6$ Monte Carlo samples. Crosses: direct simulations at $N=1024$ averaged over 100 runs; error bars show standard deviations. Good agreement is observed over a wide time window except for critical cases, which are presumably due to finite-size effects.
• Figure 2: Finite-time retrieval performance as a function of the initial overlap $m(0)$ and loading level $\alpha$ for representative interaction orders $p=3,4,7,10$. Colors represent the final overlap in runs with $T=20$, obtained from DMFT. The gradual change of color indicates slow relaxation near the dynamical retrieval boundary. The vertical dashed lines stand for the storage capacity evaluated by the RS static analysis.
• Figure 3: Overlap with the retrieved pattern after $T = 20,50,100,200$ iterations for $p = 3$. Left: DMFT prediction, obtained by solving the effective single-site equations with $10^6$ Monte Carlo samples. right: direct simulations at $N=1024$ averaged over 100 runs. The vertical dashed lines stand for the storage capacity evaluated by the RS static theory.
• Figure 4: Final overlap $m(T)$ as a function of the loading rate $\alpha$ for $p=3$ with fixed initial overlap $m(0)=0.8$. Curves are shown for $T=20,50,100,200$. As the observation time increases, the drop in $m(T)$ shifts toward smaller $\alpha$. The vertical dashed and dash-dotted lines indicate the critical loadings from the RS and 1RSB static analyses, respectively.