Temporal Complexity and Self-Organization in an Exponential Dense Associative Memory Model

TL;DR

This work investigates temporal complexity in a stochastic exponential DAM (SEDAM) with multiplicative noise, using a non-equilibrium dynamical framework that does not rely on canonical equilibrium. By analyzing birth–death transition events corresponding to neural coincidences and avalanches, the authors apply event-driven diffusion scaling (EDDiS) to extract diffusion exponents $H$ and $\\delta$ and a correlation time $\\mathcal{T}_c$, revealing extended critical regions where $H>0.5$ and $\\delta>0.5$, with $H>1$ in the long-time regime for the critical phase. The results show that complex intermittency and strong long-range temporal correlations emerge for intermediate noise levels, and that increasing memory load $K$ lowers the noise range $p$ required to reach critical TC, while sharpening the transition between phases. These findings position TC as a complementary framework for understanding learning and information processing in neural systems and highlight a link between memory load and the network’s self-organizing capacity, with implications for artificial and biological networks. $E_t = - extstyle\, abla_S E[S_t] = -\sum_{ u=1}^K \exp(\boldsymbol{\xi}_\nu^T \boldsymbol{S}_t)$ and $S^i_t = \epsilon[i,t] \cdot \text{sgn}( -\sum_{\nu=1}^K \exp(\boldsymbol{\xi}_\nu^T \boldsymbol{S}^{(i+)}_{t-1}) + \sum_{\nu=1}^K \exp(\boldsymbol{\xi}_\nu^T \boldsymbol{S}^{(i-)}_{t-1}) )$ define the model dynamics, while $\psi(\tau) \sim \tau^{-\mu}$ and the scaling laws $\text{DFA}(\Delta t) \sim (\Delta t)^{2H}$, $S(\Delta t) = A + \delta \log (\Delta t)$ describe TC analyses.

Abstract

Dense Associative Memory (DAM) models generalize the classical Hopfield model by incorporating n-body or exponential interactions that greatly enhance storage capacity. While the criticality of DAM models has been largely investigated, mainly within a statistical equilibrium picture, little attention has been devoted to the temporal self-organizing behavior induced by learning. In this work, we investigate the behavior of a stochastic exponential DAM (SEDAM) model through the lens of Temporal Complexity (TC), a framework that characterizes complex systems by intermittent transition events between order and disorder and by scale-free temporal statistics. Transition events associated with birth-death of neural avalanche structures are exploited for the TC analyses and compared with analogous transition events based on coincidence structures. We systematically explore how TC indicators depend on control parameters, i.e., noise intensity and memory load. Our results reveal that the SEDAM model exhibits regimes of complex intermittency characterized by nontrivial temporal correlations and scale-free behavior, indicating the spontaneous emergence of self-organizing dynamics. These regimes emerge in small intervals of noise intensity values, which, in agreement with the extended criticality concept, never shrink to a single critical point. Further, the noise intensity range needed to reach the critical region, where self-organizing behavior emerges, slightly decreases as the memory load increases. This study highlights the relevance of TC as a complementary framework for understanding learning and information processing in artificial and biological neural systems, revealing the link between the memory load and the self-organizing capacity of the network.

Figures (10)

• Figure 1: Coincidence events. IET autocorrelation functions with $K = 1$ (top panel) and $K = 100$ (bottom panel) stored patterns, shown across the three dynamical phases: sub-critical, critical, and super-critical (from left to right). "Lag" in the $x$-axis stands for $\Delta n$.
• Figure 2: Coincidence events. (a) DFA curves in the three different phases (sub-critical, critical, super-critical) for $K = 1, 10, 100, 1000$ stored patterns. (c) DE curves in the three different phases (sub-critical, critical, super-critical) for $K = 1, 10, 100, 1000$ stored patterns. Each panel legend shows the range of $p$ values relating to the phase indicated in the legend itself. DFA and DE are referenced to the values $DFA(\Delta t_0)$ and $DE(\Delta t_0)$, respectively, being $\Delta t_0$ the smallest time lag at which DFA and DE are computed. The reported DFA and DE functions are derived as averages (points) and standard deviations (vertical bars) computed over the range of noise intensity values $p$ reported in the figure legend. (c) Boxplots of $H$ (left) and $\delta$ (right) values in the three phases (sub-critical, critical, super-critical) for both short- and long-time lag regimes. The crossover between short- and long-time ranges decreases with $K$.
• Figure 3: Avalanche birth-death events. IET autocorrelation function with $K = 1$ (top panel) and $K = 100$ (bottom panel) stored patterns, shown across the three dynamical phases: sub-critical, critical, and super-critical (from left to right). "Lag" in the $x$-axis stands for $\Delta n$.
• Figure 4: Same as Figure \ref{['fig:H_delta_vs_phases_concidences']}, but for avalanche birth-death events.
• Figure 5: Neuron Firing Rates with $K = 1$ (top panel) and $K = 100$ (bottom panel) stored patterns, shown across the three dynamical phases: sub-critical, critical, and super-critical (from left to right).