Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Thermal Robustness of Retrieval in Dense Associative Memories: LSE vs LSR Kernels

Tatiana Petrova

Abstract

Understanding whether retrieval in dense associative memories survives thermal noise is essential for bridging zero-temperature capacity proofs with the finite-temperature conditions of practical inference and biological computation. We use Monte Carlo simulations to map the retrieval phase boundary of two continuous dense associative memories (DAMs) on the $N$-sphere with an exponential number of stored patterns $M = e^{αN}$: a log-sum-exp (LSE) kernel and a log-sum-ReLU (LSR) kernel. Both kernels share the zero-temperature critical load $α_c(0)=0.5$, but their finite-temperature behavior differs markedly. The LSE kernel sustains retrieval at arbitrarily high temperatures for sufficiently low load, whereas the LSR kernel exhibits a finite support threshold below which retrieval is perfect at any temperature; for typical sharpness values this threshold approaches $α_c$, making retrieval nearly perfect across the entire load range. We also compare the measured equilibrium alignment with analytical Boltzmann predictions within the retrieval basin.

Thermal Robustness of Retrieval in Dense Associative Memories: LSE vs LSR Kernels

Abstract

Understanding whether retrieval in dense associative memories survives thermal noise is essential for bridging zero-temperature capacity proofs with the finite-temperature conditions of practical inference and biological computation. We use Monte Carlo simulations to map the retrieval phase boundary of two continuous dense associative memories (DAMs) on the -sphere with an exponential number of stored patterns : a log-sum-exp (LSE) kernel and a log-sum-ReLU (LSR) kernel. Both kernels share the zero-temperature critical load , but their finite-temperature behavior differs markedly. The LSE kernel sustains retrieval at arbitrarily high temperatures for sufficiently low load, whereas the LSR kernel exhibits a finite support threshold below which retrieval is perfect at any temperature; for typical sharpness values this threshold approaches , making retrieval nearly perfect across the entire load range. We also compare the measured equilibrium alignment with analytical Boltzmann predictions within the retrieval basin.
Paper Structure (3 sections, 9 equations, 2 figures)

This paper contains 3 sections, 9 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Monte Carlo Simulations
  3. Summary, Discussion, and Outlook

Figures (2)

  • Figure 1: Phase diagrams for the spherical DAM with exponential capacity $M = e^{\alpha N}$: LSE (left, $\beta_{\mathrm{net}} = 1$) and LSR (right, $b=3.41$). The non-retrieval region is shown in red. The retrieval boundary lies below and to the left of the visible red boundary, corresponding to $\phi\sim 1$ at low $T$ and decreasing to $\phi\sim 0.4$ at high $T$ due to thermal broadening of the state (see main text). LSR exhibits a sharp threshold at $\alpha_{\text{th}} = (1-b^{-1})^2/2$, below which retrieval is perfect at any temperature. In both cases, retrieval breaks down at $\alpha = 0.5$ (dashed line).
  • Figure 2: Comparison of the Boltzmann equilibrium $\phi_{\rm eq}$ (solid curve) within the LSE basin with the Monte Carlo simulations at $\alpha = 0.05, 0.1$ (left), and the Boltzmann equilibrium $\phi_{\rm eq}$ within the LSR retrieval basin with the Monte Carlo simulations at $\alpha = 0.1$ (right). The dotted line shows the hard-wall threshold $\phi_c$.