Empirical universality and non-universality of local dynamics in the Sherrington-Kirkpatrick model

TL;DR

This work examines whether local optimization dynamics for the Sherrington-Kirkpatrick spin glass exhibit universality across the distribution of couplings. It contrasts greedy and reluctant local algorithms, showing universal polynomial runtime scaling for greedy dynamics but distribution-dependent scaling for reluctant dynamics, with a discrepancy-based dichotomy separating universal and non-universal cases. The study combines extensive simulations across many $\mu$ and $\lambda$, moment-matching tests, sparsity, and interpolation between discrete and continuous regimes, plus an EVT-based analysis of first-step energy changes to explain non-universality. The findings highlight how subtle arithmetic structure in the coupling distribution shapes local-search performance, with potential implications for designing optimization heuristics in high-dimensional, rough energy landscapes. The results provide a nuanced picture of universality in random optimization, identifying clear conditions under which simple dynamics can achieve robust performance and when they become sensitive to distributional details.

Abstract

Several recent works have aimed to design algorithms for optimizing the Hamiltonians of spin glass models from statistical physics. While Montanari (2018) eventually gave a sophisticated message-passing algorithm to do this nearly optimally for the Sherrington-Kirkpatrick (SK) model, Parisi (2003) observed earlier that a simple yet unusual algorithm seems to perform just as well: perform local reluctant search, repeatedly making the local adjustment improving the objective function by the smallest possible amount. This is in contrast to the more intuitive local greedy search that repeatedly makes the local adjustment improving the objective by the largest possible amount. We study empirically how the performance of these algorithms depends on the distribution of entries of the coupling matrix in the SK model. We find evidence that, while the runtime of greedy search enjoys universality over a broad range of distributions, the runtime of reluctant search surprisingly is not universal, sometimes depending quite sensitively on the entry distribution. We propose that one mechanism leading to this non-universality is a change in the behavior of reluctant search when the couplings have discrete support on an evenly-spaced grid, and give experimental results supporting this proposal and investigating other properties of a distribution that might affect the performance of reluctant search.

Figures (9)

• Figure 1: Illustration of power law fits. For various choices of $\mu$ and $\lambda$, we plot power laws fit to the runtime $T$ as a function of $N$. Dotted lines show the fitted power law on a log-log plot, and box plots show the distribution of several random draws of the runtime over 50 trials for various $N$. Darker colors indicate larger $\lambda$ (more reluctant dynamics).
• Figure 2: Fluctuations of power law fits. For various $\mu$ and $\lambda$, we plot several linear fits on a log-log plot of the relationship between $T$ and $N$, estimated from independent experiments. The lines are close to one another for each such choice, and the thickness of the resulting "bands" reflects the fluctuations in our estimates of $\widehat{\beta}$ due to random sampling of the coupling matrices.
• Figure 3: Fluctuations of exponent estimates. We plot violin plot representations of the distribution of $\widehat{\beta}(\mu, \lambda)$ estimated 50 repeated regressions using 20 independent draws of $J$ for each value of $N$, for various $\mu$ and $\lambda$.
• Figure 4: Demonstration of non-universality. For various entry distributions $\mu$, we plot $\widehat{\beta}(\mu, \lambda)$ over a range of $\lambda$ with error bars over several independent estimates of $\widehat{\beta}$. We clearly observe that for small $\lambda$ (greedy dynamics) the estimated values are similar, while for large $\lambda$ (reluctant dynamics) the estimated values are clearly different beyond errors due to our approximation procedure.
• Figure 5: Moment matching to Gaussian. For entry distributions $\mu$ matching various numbers of moments of the Gaussian distribution, we plot $\widehat{\beta}(\mu, \lambda)$ over a range of $\lambda$ with error bars over several independent estimates of $\widehat{\beta}$. The estimates for the Gaussian distribution are shown by the bold black line, and the numbers of matching moments for each distribution are given in the legend.

Theorems & Definitions (11)

• Definition 1.1
• Definition 1.2: Hamming distance
• Conjecture 1.3: Informal
• Conjecture 1.4: Informal
• Definition 1.5: Discrepancy of a probability measure
• Example 1.6: Continuous support
• Example 1.7: Integer support
• Example 1.8: Discrete support with discrepancy zero
• Conjecture 1.9: Informal
• Conjecture 1.10