Content-Addressable Memory with a Content-Free Energy Function

TL;DR

Content-addressable memory can be achieved not only by energy minima but via kinetic traps. The paper introduces a kinetic encoding framework in Hopfield-style networks where the energy depends on the global overlap as $K|m|$ while pattern information enters through unit-wise update frequencies $\omega_i$; this yields transient yet rapid pattern retrieval with capacity comparable to energy-based schemes. The authors derive retrieval thresholds $K_{ m min}$ and $Q_{ m min}$, lifetimes, and phase diagrams, and validate them with simulations, including extensions to dilute connectivity, sparse patterns, and continuous units. The work demonstrates that kinetic stability can realize high-capacity content-addressable memory in both biological and artificial contexts, with distinctive aging dynamics and a flexible design space for memory encoding.

Abstract

Content-addressable memory, i.e. stored information that can be retrieved from content-based cues, is key to computation. Besides natural and artificial neural networks, physical learning systems have recently been shown to have remarkable ability in this domain. While classical neural network models encode memories as energy minima, biochemical systems have been shown to be able to process information based on purely kinetic principles. This opens the question of whether neural networks can also encode information kinetically. Here, we propose a minimal model for content-addressable memory in which the kinetics, and not the energy function, are used to encode patterns. We find that the performance of this kinetic encoding approach is comparable to that of classical energy-based encoding schemes. This highlights the fundamental significance of the kinetic stability of kinetic traps as an alternative to the thermodynamic stability of energy minima, offering new insights into the principles of computation in physical and synthetic systems.

Figures (16)

• Figure 1: Schematics of kinetic encoding setup.A. In energetic encoding, the pattern state is preferred due to its lower energy. B. In kinetic encoding, the pattern state is discriminated from other states with the same energy via the lower kinetic barriers along its pathway. C.$N=8$ fully-connected units encode $P=3$ activity patterns. The retrieval of pattern $\mu=1$ corresponds to overlap values $(m_1,m_2,m_3)=(1,0,0)$ and activity $m=0$. D. Sketch of the energy diagram for the retrieval of pattern $\mu=1$ from a state with only $-1$ errors (black minus signs) besides the cue (green plus and minus signs). Going down in energy, the system can either create a $+1$ error (left transition) or correct a $-1$ error (right transition), the latter being kinetically favored.
• Figure 2: Successful retrieval of a single pattern.A. Stochastic trajectories for $P=1$ and $N=2^{10}$, starting from a random state with $m_1=0.2$ and $m=-0.8$; see [SM Sec. \ref{['sec:simu_details']}] for simulation details. At a fixed value $K=10$, $m_1$ evolves toward a plateau whose value $m_1^*$ increases with $Q$. B. At $Q=10$, the plateau value increases with $K$. The small fluctuations around the plateau value disappear with increasing $K$. C. Phase diagram of pattern retrieval. The plateau value $m_1^*$ corresponds to a $1\%$ error for $Q$ and $K$ above the analytical estimates $\approx6.9$ and $4.6$ respectively (blue and red lines) [SM Sec. \ref{['sec:theory']}]. The grey regions refer to the time trajectories shown in A-B. D. The blue trajectories from A have plateau values $m^*\approx0$ (larger squares), while for the red trajectories from B, $m_1^*\approx1+m^*$.
• Figure 3: Large kinetic stability and high encoding capacity.A. The system eventually escapes the kinetic trap ($m_1=1$) toward states with lower values of $m_1$. The retrieval dynamics follows the exponential prediction (red line). The escape dynamics rescaled horizontally by the lifetime, $t/\tau_{\rm life}$, is similar for different values of $K,Q,N$ with a slope of $\tfrac{1}{2}$ in log-log (black line). Additionally, the minimum error decreases with increasing values of $K$ and $Q$. Here, $P=1$. B. The trajectories in log time show that the eventual slow decrease of $m_1$ occurs near the $m=0$ line. The values at time $t=10^3$ (dashed line in A) are shown as larger squares. C. The plateau value of the overlap with the cued pattern, $m_1^*$, decreases with increasing number of patterns, $P$ [SM Sec. \ref{['sec:capa']}]. For Hebbian pair couplings, the encoding capacity $P_{\rm max}$ at which $m_1^*=0.95$ scales with $N$. The capacity is higher for $m_1(0)=0.9$ (brown) than for $m_1(0)=0.2$ (orange), but still far from the Gardner bound Gardner_JPhysA88_2Shim93 (dashed line). By contrast, for three-body couplings, $P_{\rm max}$ scales with $N^2$ (pink).
• Figure 4: Dilute couplings and sparse patterns.A. For dilute connections between units, the encoding capacity $P_{\rm max}$ decreases linearly with the dilution. B. For patterns with a vanishing fraction of inactive units, $P_{\rm max}$ diverges.
• Figure 5: Temporal correlations are not time-translation invariant. Beginning from the pattern state at $t=0$, the correlation between states at times $t_0$ and $t+t_0$ depends on the waiting time $t_0$. For $t_0\gtrsim \tau_{\rm life}$, $C(t,t_0)$ abruptly drops for $t\sim \tau_m$, whereas $C(t,0)$ decreases over timescale $\tau_{\rm life}\gg \tau_m$. Here, $N=100$, $P=1$, $m_1(0)=1$ and $K=Q=6$ such that the lifetime at which $m_1=0.8$ is $\tau_{\rm life}\approx1200$, while $C(\tau_m,1900)=0.8$ for $\tau_m\approx 100$. Inset: Timescale $\tau_m$ increases exponentially with the energetic drive, $K$.