Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stable algorithms cannot reliably find isolated perceptron solutions

Shuyang Gong, Brice Huang, Shuangping Li, Mark Sellke

Abstract

We study the binary perceptron, a random constraint satisfaction problem that asks to find a Boolean vector in the intersection of independently chosen random halfspaces. A striking feature of this model is that at every positive constraint density, it is expected that a $1-o_N(1)$ fraction of solutions are \emph{strongly isolated}, i.e. separated from all others by Hamming distance $Ω(N)$. At the same time, efficient algorithms are known to find solutions at certain positive constraint densities. This raises a natural question: can any isolated solution be algorithmically visible? We answer this in the negative: no algorithm whose output is stable under a tiny Gaussian resampling of the disorder can \emph{reliably} locate isolated solutions. We show that any stable algorithm has success probability at most $\frac{3\sqrt{17}-9}{4}+o_N(1)\leq 0.84233$. Furthermore, every stable algorithm that finds a solution with probability $1-o_N(1)$ finds an isolated solution with probability $o_N(1)$. The class of stable algorithms we consider includes degree-$D$ polynomials up to $D\leq o(N/\log N)$; under the low-degree heuristic \cite{hopkins2018statistical}, this suggests that locating strongly isolated solutions requires running time $\exp(\widetildeΘ(N))$. Our proof does not use the overlap gap property. Instead, we show via Pitt's correlation inequality that after a random perturbation of the disorder, the number of solutions located close to a pre-existing isolated solution cannot concentrate at $1$.

Stable algorithms cannot reliably find isolated perceptron solutions

Abstract

We study the binary perceptron, a random constraint satisfaction problem that asks to find a Boolean vector in the intersection of independently chosen random halfspaces. A striking feature of this model is that at every positive constraint density, it is expected that a fraction of solutions are \emph{strongly isolated}, i.e. separated from all others by Hamming distance . At the same time, efficient algorithms are known to find solutions at certain positive constraint densities. This raises a natural question: can any isolated solution be algorithmically visible? We answer this in the negative: no algorithm whose output is stable under a tiny Gaussian resampling of the disorder can \emph{reliably} locate isolated solutions. We show that any stable algorithm has success probability at most . Furthermore, every stable algorithm that finds a solution with probability finds an isolated solution with probability . The class of stable algorithms we consider includes degree- polynomials up to ; under the low-degree heuristic \cite{hopkins2018statistical}, this suggests that locating strongly isolated solutions requires running time . Our proof does not use the overlap gap property. Instead, we show via Pitt's correlation inequality that after a random perturbation of the disorder, the number of solutions located close to a pre-existing isolated solution cannot concentrate at .

Paper Structure

This paper contains 30 sections, 12 theorems, 114 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Perceptron models
  3. Satisfiability threshold.
  4. Algorithms.
  5. Strong freezing.
  6. Why algorithms may still succeed.
  7. Main results: algorithmic hardness of locating isolated solutions
  8. Stable algorithms cannot find isolated solutions
  9. High-success algorithms avoid isolated solutions
  10. Low-degree consequence
  11. Related work
  12. Overlap gap property (OGP)
  13. Low-degree polynomials
  14. Technical overview
  15. Stable algorithms cannot find isolated solutions.
  16. ...and 15 more sections

Key Result

Theorem 1.1

Fix any $\kappa\in{\mathbb R}$, $\alpha>0$, and $\iota\in(0,1)$. Let $\eta_N = o_N(1)$ satisfy Let $\mathcal{A}_N$ be $(\rho_N,t_N)$-stable at noise level $\eta_N$ with $\rho_N=o(\sqrt{N}),\,t_N=o_N(1).$ Then $\blacktriangleleft$$\blacktriangleleft$

Figures (1)

  • Figure 1: Left: the ball $\mathsf B_{\sqrt{\iota N}/3,2}\bigl(\mathcal{A}_N({\boldsymbol G})\bigr)$ around the output $\mathcal{A}_N({\boldsymbol G})$, together with a larger neighborhood around $\mathcal{A}_N(\widetilde{{\boldsymbol G}})$. Right: schematically, the ball $\mathsf B_{\sqrt{\iota N}/3,2}\bigl(\mathcal{A}_N({\boldsymbol G})\bigr)$ is divided into a left region containing ${\boldsymbol{\sigma}}$ and a right region containing ${\boldsymbol{\tau}}$.

Theorems & Definitions (30)

  • Definition 1: Isolated solution
  • Definition 2: Margin
  • Definition 3: $\ell_2$-stability under resampling
  • Definition 4: Locating solutions in $\ell_2$
  • Theorem 1.1
  • Remark 1.2: Where the constant $\frac{3\sqrt{17}-9}{4}$ comes from
  • Remark 1.3: On the stability condition
  • Theorem 1.4
  • Remark 1.5
  • Definition 5: Degree-$D$ polynomial output with bounded second moment
  • ...and 20 more