Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rapid mixing in positively weighted restricted Boltzmann machines

Weiming Feng, Heng Guo, Minji Yang

Abstract

We show polylogarithmic mixing time bounds for the alternating-scan sampler for positively weighted restricted Boltzmann machines. This is done via analysing the same chain and the Glauber dynamics for ferromagnetic two-spin systems, where we obtain new mixing time bounds up to the critical thresholds.

Rapid mixing in positively weighted restricted Boltzmann machines

Abstract

We show polylogarithmic mixing time bounds for the alternating-scan sampler for positively weighted restricted Boltzmann machines. This is done via analysing the same chain and the Glauber dynamics for ferromagnetic two-spin systems, where we obtain new mixing time bounds up to the critical thresholds.

Paper Structure

This paper contains 30 sections, 45 theorems, 218 equations, 1 algorithm.

Table of Contents

  1. Introduction
  2. Ferromagnetic two-spin systems
  3. Proof overview
  4. All-to-one influence bound
  5. Mixing from typical-case ASSM
  6. Typical-case ASSM for ferro spin systems
  7. Preliminaries
  8. Markov chain and mixing time
  9. Self-reducibility
  10. Self-avoiding walk tree
  11. Tree recursion and potential function
  12. All-to-one influence bound
  13. General influence decay results
  14. Proof of the influence bound
  15. Mixing from typical-case aggregate strong spatial mixing
  16. ...and 15 more sections

Key Result

Theorem 1

Let $c > 0$ be an arbitrary constant. For any restricted Boltzmann machine $(W,\theta)$ with $n$ variables, if for all $u,v$, either $w_{uv} \geq c$ or $w_{uv}=0$, and for all $v \in V$, $\theta_v \ge 0$, then the alternating-scan sampler over the Gibbs distribution $\mu$ of the RBM has mixing time

Theorems & Definitions (90)

  • Theorem 1
  • Definition 2: $(\beta,\gamma,\lambda)$-ferromagnetic two-spin systems
  • Theorem 3
  • Theorem 4
  • Theorem 5
  • Remark
  • Proposition 6: GuoKZ18
  • Remark
  • Proposition 7: GuoKZ18
  • Remark
  • ...and 80 more