Loop corrections for hard spheres in Hamming space

TL;DR

This paper tackles the problem of determining the maximum packing density of hard spheres in a binary Hamming space by formulating an exact entropy expression via an extended Belief Propagation framework that introduces auxiliary internal messages to capture loopy interactions on a tree-based graph. The authors construct a precise tree representation and derive BP equations for an extended set of variables, then explore a hierarchy of approximate BP marginals, including naive homogeneous and permutation-symmetric inhomogeneous forms. They show that these BP-based approaches reproduce the Gilbert-Varshamov lower bound for the packing density in the asymptotic limit, and that inhomogeneous solutions tend to converge to the homogeneous one as the number of spheres grows, although they cannot rule out densities above GV. The work suggests that maximizing Bethe entropy within tractable marginals could yield rigorous upper bounds and motivates exploring alternative trial marginals or entropy-based bounds to sharpen the understanding of high-dimensional sphere packing in Hamming spaces.

Abstract

We begin with an exact expression for the entropy of a system of hard spheres within the Hamming space. This entropy relies on probability marginals, which are determined by an extended set of Belief Propagation (BP) equations. The BP probability marginals are functions of auxiliary variables which are introduced to model the effects of loopy interactions on a tree-structured interaction graph. We explore various reasonable and approximate probability distributions, ensuring they align with the exact solutions of the BP equations. Our approach is based on an ansatz of (in)homogeneous cavity marginals respecting the permutation symmetry of the problem. Through thorough analysis, we aim to minimize errors in the BP equations. Our findings support the conjecture that the maximum packing density asymptotically conforms to the lower bound proposed by Gilbert and Varshamov, further validated by the solution of the loopy BP equations.

Figures (4)

• Figure 1: Illustration of a factor graph with the variables, constraints, and messages. The original variables are the sphere positions $\vec{\sigma}_i$ besides the auxiliary variables (internal messages) $h_{i\to j}$ which are to represent the effect of loopy interactions. The cavity probability marginals (external BP messages) of the extended set of variables are $\mu_{i\to j}(\vec{\sigma}_i,h_{i\to j})$. For simplicity, in this study we shall work with a chain factor graph (Panel (b)).
• Figure 2: An illustration of the geometrical measures defined by a few hard spheres of diameter $d$. The solid discs display the hard spheres. The empty discs show the space which can not be occupied by the center of another hard sphere. Variable $l_{ij}$ denotes the distance between two spheres $i$ and $j$ with overlap $O_{ij}(l_{ij})$. Distance of a point in space from sphere $i$ is denoted by $l_i$.
• Figure 3: Deviation from the uniform solution when we start from one sphere at origin. The probabilities $p_i(\vec{\sigma}_i)$ are computed numerically from the BP equations with the inhomogeneous ansatz for the cavity marginals. We assume that the other probabilities $c_{i}(\{l_j\})$ are concentrated on a single configuration $\{l_j^*\}$. The reported deviation $\Delta P=\sum_{\vec{\sigma}}|p_i(\vec{\sigma})-\frac{1}{2^n}|$ is maximized over all possible configurations of $\{l_j^*\}$.
• Figure 4: The overlap $o=O/V_d$ of two spheres of scaled diameter $\delta$ at distance $r'$. Numerical computation of this quantity shows that it is exponentially small for $\delta <1/2$.