Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sum-of-Squares & Gaussian Processes I: Certification

Juspreet Singh Sandhu, Jonathan Shi

Abstract

We introduce a class of distributions which may be considered as a smoothed probabilistic version of the ultrametric property that famously characterizes the Gibbs distributions of various spin glass models. This class of \emph{high-entropy step} (HES) distributions is expressive enough to capture a distribution achieving near-optimal average energy on spin glass models in the so-called full Replica-Symmetry Breaking (fRSB) regime. Simultaneously, with high probability, there are polynomial-size certificates on the average energy achievable by \emph{any} HES distribution which are tight within a constant factor. These certificates can be found in polynomial time by a semidefinite program corresponding to a sum-of-squares (SoS) hierarchy we introduce, termed the HES SoS hierarchy. This improves over classical sum-of-squares certificates which are loose by a factor of $n^{\lfloor p/2 - 1\rfloor/2}$.

Sum-of-Squares & Gaussian Processes I: Certification

Abstract

We introduce a class of distributions which may be considered as a smoothed probabilistic version of the ultrametric property that famously characterizes the Gibbs distributions of various spin glass models. This class of \emph{high-entropy step} (HES) distributions is expressive enough to capture a distribution achieving near-optimal average energy on spin glass models in the so-called full Replica-Symmetry Breaking (fRSB) regime. Simultaneously, with high probability, there are polynomial-size certificates on the average energy achievable by \emph{any} HES distribution which are tight within a constant factor. These certificates can be found in polynomial time by a semidefinite program corresponding to a sum-of-squares (SoS) hierarchy we introduce, termed the HES SoS hierarchy. This improves over classical sum-of-squares certificates which are loose by a factor of .
Paper Structure (169 sections, 85 theorems, 452 equations, 2 algorithms)

This paper contains 169 sections, 85 theorems, 452 equations, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. SoS proofs for HES distributions
  4. Spherical spin glass model
  5. Ensemble-agnosticism
  6. SoS HES hierarchy on the hypercube
  7. SoS proofs for analytic inequalities
  8. Implications for Sum-of-Squares relaxations
  9. Convex relaxations over distributions
  10. Concentration of measure in SoS
  11. Robustness in average-case
  12. Entropy-energy variational trade-off
  13. Average case optimization in SoS
  14. Related work
  15. Analytic & geometric properties of spin glasses
  16. ...and 154 more sections

Key Result

Theorem 1.1

Let $\mathcal{A}$ be some set of polynomial axioms that can be satisfied by the moments of a sequence of step variables $\sigma_i = \frac{1}{\sqrt{k}}\left(v_1 + \dots + v_i\right)$. Let $H$ be a function on the sphere $\mathcal{S}^{n-1}(1)$ in $\varmathbb{R}^n$, for which there exists an extension Then the HES SoS hierarchy outputs a certificate of value $(1-O(\varepsilon))e^{O(k^2)}(\mathcal{E}

Theorems & Definitions (202)

  • Theorem 1.1: Certifiable properties of HES distributions
  • Definition 1.2: High-Entropy Step (HES) Distributions
  • Lemma 1.3: Nuclear norm property of HES moments (informal)
  • Theorem 1.4
  • Lemma 1.5: HES hierarchy on ansiotropic spherical spin-glasses
  • Conjecture 1.6: Ensemble-agnosticism
  • Conjecture 1.7: Hessians of smooth Gaussian processes
  • Definition 2.1: Mixed Spherical $p$-Spin Glass
  • Definition 2.2: Mixture Polynomial
  • Definition 3.1: SoS Axioms $\mathcal{A}$
  • ...and 192 more