Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sinkhorn Based Associative Memory Retrieval Using Spherical Hellinger Kantorovich Dynamics

Aratrika Mustafi, Soumya Mukherjee

Abstract

We propose a dense associative memory for empirical measures (weighted point clouds). Stored patterns and queries are finitely supported probability measures, and retrieval is defined by minimizing a Hopfield-style log-sum-exp energy built from the debiased Sinkhorn divergence. We derive retrieval dynamics as a spherical Hellinger Kantorovich (SHK) gradient flow, which updates both support locations and weights. Discretizing the flow yields a deterministic algorithm that uses Sinkhorn potentials to compute barycentric transport steps and a multiplicative simplex reweighting. Under local separation and PL-type conditions we prove basin invariance, geometric convergence to a local minimizer, and a bound showing the minimizer remains close to the corresponding stored pattern. Under a random pattern model, we further show that these Sinkhorn basins are disjoint with high probability, implying exponential capacity in the ambient dimension. Experiments on synthetic Gaussian point-cloud memories demonstrate robust recovery from perturbed queries versus a Euclidean Hopfield-type baseline.

Sinkhorn Based Associative Memory Retrieval Using Spherical Hellinger Kantorovich Dynamics

Abstract

We propose a dense associative memory for empirical measures (weighted point clouds). Stored patterns and queries are finitely supported probability measures, and retrieval is defined by minimizing a Hopfield-style log-sum-exp energy built from the debiased Sinkhorn divergence. We derive retrieval dynamics as a spherical Hellinger Kantorovich (SHK) gradient flow, which updates both support locations and weights. Discretizing the flow yields a deterministic algorithm that uses Sinkhorn potentials to compute barycentric transport steps and a multiplicative simplex reweighting. Under local separation and PL-type conditions we prove basin invariance, geometric convergence to a local minimizer, and a bound showing the minimizer remains close to the corresponding stored pattern. Under a random pattern model, we further show that these Sinkhorn basins are disjoint with high probability, implying exponential capacity in the ambient dimension. Experiments on synthetic Gaussian point-cloud memories demonstrate robust recovery from perturbed queries versus a Euclidean Hopfield-type baseline.
Paper Structure (36 sections, 20 theorems, 296 equations, 2 figures)

This paper contains 36 sections, 20 theorems, 296 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Setting and notations
  3. Theoretical Guarantees
  4. Conclusion
  5. First variation of the Sinkhorn divergence
  6. First variation of the log-sum-exp energy
  7. From entropic potentials to deterministic barycentric maps
  8. Spherical Hellinger-Kantorovich gradient flow (Continuous-time)
  9. Transport$+$reaction based continuity equation
  10. Riemmanian structure induced by Spherical Hellinger-Kantorovich geometry on the space of probability measures
  11. The manifold of finitely supported discrete measures equipped with SHK geometry
  12. Defining an appropriate retraction map for $\mathcal{M}_M$
  13. Riemannian gradient of the energy functional $E$ on $\mathcal{M}_M$
  14. Differential of $E$ in particle coordinates :
  15. Riemannian gradient of E :
  16. ...and 21 more sections

Key Result

Theorem 1

Fix $d \geq 1, M \geq 2$, $a_{\min} >0$, $\Delta_{\min} > 0$ and let $\Omega \subset \mathbb{R}^d$ be open, bounded and convex. Assume there exists $c \in \Omega$ and $R>0$ such that the closed ball $\bar{\mathbb{B}}(c, R) \subset \Omega$. Fix any $\sigma \in(0, R / 4)$, such that $\mathcal{Z}_{\sig Let $X_1,\dots,X_N$ be generated by the sampling mechanism described as SampAlgo (see Sec SampAlgo)

Figures (2)

  • Figure 1: Experiment 1: Sinkhorn Algo and Euclidean Algo are both able to retrieve correct patterns from noisy queries
  • Figure 2: Experiment 2: Sinkhorn Algo succeeds in retrieving correct patterns from noisy queries in all instances, but Euclidean Algo fails in 3 cases.

Theorems & Definitions (40)

  • Theorem 1: Exponential storage capacity and high probability separation of patterns
  • Theorem 2: Geometric convergence in Sinkhorn divergence to the local minimizer and local basin invariance of gradient descent iterates
  • Theorem 3: Sinkhorn distance between minimizer in local basin and stored pattern and stability of stored pattern
  • proof
  • proof
  • Lemma 1: local basin compactness inside parameter space
  • proof
  • Lemma 2: uniform retraction domain and local metric upper bound
  • proof
  • proof
  • ...and 30 more