Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Computationally sufficient statistics for Ising models

Abhijith Jayakumar, Shreya Shukla, Marc Vuffray, Andrey Y. Lokhov, Sidhant Misra

TL;DR

This work tackles the problem of learning Ising-model Gibbs distributions when only limited statistics are observable, linking computational tractability to observational power. It introduces a polynomially-approximated gradient for the Interaction Screening Estimator, enabling convex optimization using low-order moments and establishing that statistics of order $O(\gamma)$ suffice for accurate parameter, structure, and magnetic-field recovery, with a sample complexity that scales as $O\big(e^{8\gamma}\gamma^4\log p / \epsilon^4\big)$. The paper also shows that structure can be recovered via thresholding under a separation condition, and that magnetic fields can be learned after fixing the structure; with stronger priors (e.g., known $D$-regular graphs), even lower-order statistics become sufficient. These results reveal a concrete information-computation tradeoff in learning discrete graphical models and provide practical algorithms for scenarios where full configurations are unavailable, with potential extensions to physics-inspired priors and more constrained model classes.

Abstract

Learning Gibbs distributions using only sufficient statistics has long been recognized as a computationally hard problem. On the other hand, computationally efficient algorithms for learning Gibbs distributions rely on access to full sample configurations generated from the model. For many systems of interest that arise in physical contexts, expecting a full sample to be observed is not practical, and hence it is important to look for computationally efficient methods that solve the learning problem with access to only a limited set of statistics. We examine the trade-offs between the power of computation and observation within this scenario, employing the Ising model as a paradigmatic example. We demonstrate that it is feasible to reconstruct the model parameters for a model with $\ell_1$ width $γ$ by observing statistics up to an order of $O(γ)$. This approach allows us to infer the model's structure and also learn its couplings and magnetic fields. We also discuss a setting where prior information about structure of the model is available and show that the learning problem can be solved efficiently with even more limited observational power.

Computationally sufficient statistics for Ising models

TL;DR

This work tackles the problem of learning Ising-model Gibbs distributions when only limited statistics are observable, linking computational tractability to observational power. It introduces a polynomially-approximated gradient for the Interaction Screening Estimator, enabling convex optimization using low-order moments and establishing that statistics of order suffice for accurate parameter, structure, and magnetic-field recovery, with a sample complexity that scales as . The paper also shows that structure can be recovered via thresholding under a separation condition, and that magnetic fields can be learned after fixing the structure; with stronger priors (e.g., known -regular graphs), even lower-order statistics become sufficient. These results reveal a concrete information-computation tradeoff in learning discrete graphical models and provide practical algorithms for scenarios where full configurations are unavailable, with potential extensions to physics-inspired priors and more constrained model classes.

Abstract

Learning Gibbs distributions using only sufficient statistics has long been recognized as a computationally hard problem. On the other hand, computationally efficient algorithms for learning Gibbs distributions rely on access to full sample configurations generated from the model. For many systems of interest that arise in physical contexts, expecting a full sample to be observed is not practical, and hence it is important to look for computationally efficient methods that solve the learning problem with access to only a limited set of statistics. We examine the trade-offs between the power of computation and observation within this scenario, employing the Ising model as a paradigmatic example. We demonstrate that it is feasible to reconstruct the model parameters for a model with width by observing statistics up to an order of . This approach allows us to infer the model's structure and also learn its couplings and magnetic fields. We also discuss a setting where prior information about structure of the model is available and show that the learning problem can be solved efficiently with even more limited observational power.
Paper Structure (27 sections, 14 theorems, 107 equations, 2 algorithms)

This paper contains 27 sections, 14 theorems, 107 equations, 2 algorithms.

Table of Contents

  1. Introduction
  2. Further Related Work
  3. Learning from statistical queries.
  4. Mean-field heuristics.
  5. Continuous variables and quantum generalizations.
  6. Notation:
  7. Interaction screening with approximate gradients
  8. Proof of Theorem \ref{['thm:main_thm']}.
  9. Structure learning
  10. Learning magnetic fields
  11. Learning under stronger structural priors
  12. Conclusion
  13. Technical Lemmas
  14. Proof of Lemma \ref{['lem:poly_exp']}
  15. Proof of Lemma \ref{['lem:finte_sample_1']}
  16. ...and 12 more sections

Key Result

Theorem 1

Suppose we want to achieve error $\epsilon \leq 4\gamma$ on the $\theta^*_{u,v}$ parameters in the following metric, Then Algorithm alg:ais-moments, with parameters $T =\left \lceil\frac{576 \ p \gamma^2 e^{8\gamma} (1 + \gamma)^2}{\epsilon^4} \right\rceil$, $\eta = \frac{2 \gamma e^{-\gamma}}{3 \sqrt{pT}}$, data in the form of $S_{d+2-(d\bmod 2),n}$ with $d = \left\lceil \frac{3 \gamma + \log(\f

Theorems & Definitions (22)

  • Theorem 1
  • Lemma 1: Statistical approximation error
  • Lemma 2: Polynomial approximation error
  • Lemma 3: Robustness of gradient descent
  • Lemma 4: Bounded gradient
  • proof
  • Lemma 5
  • Theorem 2: Structure recovery by thresholding
  • Theorem 3: Magnetic field learning
  • proof
  • ...and 12 more