Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Potential Hessian Ascent: The Sherrington-Kirkpatrick Model

David Jekel, Juspreet Singh Sandhu, Jonathan Shi

TL;DR

This work presents the first iterative spectral algorithm to find near-optimal solutions for a random quadratic objective over the discrete hypercube, and lays the groundwork for (possibly) demonstrating low-degree sum-of-squares certificates over high-entropy step distributions for a relaxed version of the Parisi formula.

Abstract

We present the first iterative spectral algorithm to find near-optimal solutions for a random quadratic objective over the discrete hypercube, resolving a conjecture of Subag [Subag, Communications on Pure and Applied Mathematics, 74(5), 2021]. The algorithm is a randomized Hessian ascent in the solid cube, with the objective modified by subtracting an instance-independent potential function [Chen et al., Communications on Pure and Applied Mathematics, 76(7), 2023]. Using tools from free probability theory, we construct an approximate projector into the top eigenspaces of the Hessian, which serves as the covariance matrix for the random increments. With high probability, the iterates' empirical distribution approximates the solution to the primal version of the Auffinger-Chen SDE [Auffinger et al., Communications in Mathematical Physics, 335, 2015]. The per-iterate change in the modified objective is bounded via a Taylor expansion, where the derivatives are controlled through Gaussian concentration bounds and smoothness properties of a semiconcave regularization of the Fenchel-Legendre dual to the Parisi PDE. These results lay the groundwork for (possibly) demonstrating low-degree sum-of-squares certificates over high-entropy step distributions for a relaxed version of the Parisi formula [Open Question 1.8, arXiv:2401.14383].

Potential Hessian Ascent: The Sherrington-Kirkpatrick Model

TL;DR

This work presents the first iterative spectral algorithm to find near-optimal solutions for a random quadratic objective over the discrete hypercube, and lays the groundwork for (possibly) demonstrating low-degree sum-of-squares certificates over high-entropy step distributions for a relaxed version of the Parisi formula.

Abstract

We present the first iterative spectral algorithm to find near-optimal solutions for a random quadratic objective over the discrete hypercube, resolving a conjecture of Subag [Subag, Communications on Pure and Applied Mathematics, 74(5), 2021]. The algorithm is a randomized Hessian ascent in the solid cube, with the objective modified by subtracting an instance-independent potential function [Chen et al., Communications on Pure and Applied Mathematics, 76(7), 2023]. Using tools from free probability theory, we construct an approximate projector into the top eigenspaces of the Hessian, which serves as the covariance matrix for the random increments. With high probability, the iterates' empirical distribution approximates the solution to the primal version of the Auffinger-Chen SDE [Auffinger et al., Communications in Mathematical Physics, 335, 2015]. The per-iterate change in the modified objective is bounded via a Taylor expansion, where the derivatives are controlled through Gaussian concentration bounds and smoothness properties of a semiconcave regularization of the Fenchel-Legendre dual to the Parisi PDE. These results lay the groundwork for (possibly) demonstrating low-degree sum-of-squares certificates over high-entropy step distributions for a relaxed version of the Parisi formula [Open Question 1.8, arXiv:2401.14383].
Paper Structure (61 sections, 76 theorems, 479 equations, 1 table, 2 algorithms)

This paper contains 61 sections, 76 theorems, 479 equations, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Overview
  3. Background
  4. The Sherrington-Kirkpatrick model
  5. Parisi formula and Auffinger-Chen representation
  6. Algorithms to find the maximizers
  7. Main Results
  8. Concept: Potential Hessian Ascent
  9. Main results on the algorithm
  10. Ingredients in the analysis
  11. Significance
  12. Primal Parisi PDE
  13. Smoothness of the Parisi solution
  14. Smoothness of the convex conjugates
  15. Primal Parisi PDE and primal Auffinger-Chen SDE
  16. ...and 46 more sections

Key Result

Theorem 1.1

For any inverse temperature $\beta = \frac{1}{T} > 0$, the free energy $\mathcal{F}_{n, \beta}$ of the Sherrington-Kirkpatrick Hamiltonian $H_{\textsf{SK}}$ is given as, where $F_\mu(t) = \mu([0,t])$ is the cumulative distribution function of $\mu$ and where $\Phi_\mu(0,0)$ is the initial value of the free-entropy term that comes by solving the parabolic PDE with the terminal condition

Theorems & Definitions (138)

  • Theorem 1.1: Parisi Variational Principle, parisi1980sequencetalagrand2006parisi
  • Corollary 1.2: Ground State Energy of the Sherrington-Kirkpatrick Model
  • Theorem 1.3: Auffinger-Chen Principle, auffinger2015parisimontanari2021optimization
  • Theorem 1.5: PHA is a PTAS for the SK model
  • Proposition 2.1: Convexity and smoothness for $\Phi$ auffinger2015parisi and auffinger2015properties
  • Proposition 2.2: Refined bounds for $\partial_x \Phi$ and $\partial_{x,x}\Phi$
  • Lemma 2.3: See jagannath2016dynamic
  • Lemma 2.4: MGF estimate for AC solution
  • proof
  • proof : Proof of prop: Phi derivative bound
  • ...and 128 more