Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Binary perceptron computational gap -- a parametric fl RDT view

Mihailo Stojnic

TL;DR

A particular parametric utilization offully lifted random duality theory (fl RDT) is considered and its potential ABP's algorithmic implications are studied and a remarkable structural parametric change is uncovered as one progresses through fl RDT lifting levels.

Abstract

Recent studies suggest that asymmetric binary perceptron (ABP) likely exhibits the so-called statistical-computational gap characterized with the appearance of two phase transitioning constraint density thresholds: \textbf{\emph{(i)}} the \emph{satisfiability threshold} $α_c$, below/above which ABP succeeds/fails to operate as a storage memory; and \textbf{\emph{(ii)}} \emph{algorithmic threshold} $α_a$, below/above which one can/cannot efficiently determine ABP's weight so that it operates as a storage memory. We consider a particular parametric utilization of \emph{fully lifted random duality theory} (fl RDT) [85] and study its potential ABP's algorithmic implications. A remarkable structural parametric change is uncovered as one progresses through fl RDT lifting levels. On the first two levels, the so-called $\c$ sequence -- a key parametric fl RDT component -- is of the (natural) decreasing type. A change of such phenomenology on higher levels is then connected to the $α_c$ -- $α_a$ threshold change. Namely, on the second level concrete numerical values give for the critical constraint density $α=α_c\approx 0.8331$. While progressing through higher levels decreases this estimate, already on the fifth level we observe a satisfactory level of convergence and obtain $α\approx 0.7764$. This allows to draw two striking parallels: \textbf{\emph{(i)}} the obtained constraint density estimate is in a remarkable agrement with range $α\in (0.77,0.78)$ of clustering defragmentation (believed to be responsible for failure of locally improving algorithms) [17,88]; and \textbf{\emph{(ii)}} the observed change of $\c$ sequence phenomenology closely matches the one of the negative Hopfield model for which the existence of efficient algorithms that closely approach similar type of threshold has been demonstrated recently [87].

Binary perceptron computational gap -- a parametric fl RDT view

TL;DR

A particular parametric utilization offully lifted random duality theory (fl RDT) is considered and its potential ABP's algorithmic implications are studied and a remarkable structural parametric change is uncovered as one progresses through fl RDT lifting levels.

Abstract

Recent studies suggest that asymmetric binary perceptron (ABP) likely exhibits the so-called statistical-computational gap characterized with the appearance of two phase transitioning constraint density thresholds: \textbf{\emph{(i)}} the \emph{satisfiability threshold} , below/above which ABP succeeds/fails to operate as a storage memory; and \textbf{\emph{(ii)}} \emph{algorithmic threshold} , below/above which one can/cannot efficiently determine ABP's weight so that it operates as a storage memory. We consider a particular parametric utilization of \emph{fully lifted random duality theory} (fl RDT) [85] and study its potential ABP's algorithmic implications. A remarkable structural parametric change is uncovered as one progresses through fl RDT lifting levels. On the first two levels, the so-called sequence -- a key parametric fl RDT component -- is of the (natural) decreasing type. A change of such phenomenology on higher levels is then connected to the -- threshold change. Namely, on the second level concrete numerical values give for the critical constraint density . While progressing through higher levels decreases this estimate, already on the fifth level we observe a satisfactory level of convergence and obtain . This allows to draw two striking parallels: \textbf{\emph{(i)}} the obtained constraint density estimate is in a remarkable agrement with range of clustering defragmentation (believed to be responsible for failure of locally improving algorithms) [17,88]; and \textbf{\emph{(ii)}} the observed change of sequence phenomenology closely matches the one of the negative Hopfield model for which the existence of efficient algorithms that closely approach similar type of threshold has been demonstrated recently [87].

Paper Structure

This paper contains 19 sections, 2 theorems, 66 equations, 4 tables.

Table of Contents

  1. Introduction
  2. Analytical difficulties
  3. Algorithmic difficulties
  4. Relevant prior work
  5. Our contributions
  6. Mathematical ABP preliminaries
  7. Capacity relevance
  8. Random feasibility -- free energy connection
  9. Technical preliminaries for fitting ABP into sfl RDT framework
  10. Practical implementation of ABP -- sfl RDT connection
  11. ${\mathcal{X}}$, ${\mathcal{Y}}$, $f({\bf y})$, and $s$ specializations
  12. Numerical evaluations $(r\in\{1,2\})$ -- sfl RDT satisfiability threshold
  13. $r=1$ -- first level of lifting
  14. $r=2$ -- second level of lifting
  15. Numerical evaluations $(r\geq 3)$ -- parametric sfl RDT algorithmic threshold implications
  16. ...and 4 more sections

Key Result

Theorem 1

Stojnicbinperflrdt23Stojnicflrdt23 Consider large $n$ linear regime with $\alpha=\lim_{n\rightarrow\infty} \frac{m}{n}$ remaining constant as $n$ grows. Let $G\in{\mathbb R}^{m\times n}$ be comprised of independent standard normals and let ${\mathcal{X}}\subseteq {\mathbb S}^n$ and ${\mathcal{Y}}\su Let $\hat{{\bf p}_0}\rightarrow 1$, $\hat{{\bf q}_0}\rightarrow 1$, and $\hat{{\bf c}_0}\rightarrow

Theorems & Definitions (4)

  • Theorem 1
  • proof
  • Theorem 2
  • proof