Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Selecting Optimal Variable Order in Autoregressive Ising Models

Shiba Biswal, Marc Vuffray, Andrey Y. Lokhov

TL;DR

This work proposes to learn the Markov random field describing the underlying data, and use the inferred graphical model structure to construct optimized variable orderings that yield higher-fidelity generated samples compared to naive variable orderings.

Abstract

Autoregressive models enable tractable sampling from learned probability distributions, but their performance critically depends on the variable ordering used in the factorization via complexities of the resulting conditional distributions. We propose to learn the Markov random field describing the underlying data, and use the inferred graphical model structure to construct optimized variable orderings. We illustrate our approach on two-dimensional image-like models where a structure-aware ordering leads to restricted conditioning sets, thereby reducing model complexity. Numerical experiments on Ising models with discrete data demonstrate that graph-informed orderings yield higher-fidelity generated samples compared to naive variable orderings.

Selecting Optimal Variable Order in Autoregressive Ising Models

TL;DR

This work proposes to learn the Markov random field describing the underlying data, and use the inferred graphical model structure to construct optimized variable orderings that yield higher-fidelity generated samples compared to naive variable orderings.

Abstract

Autoregressive models enable tractable sampling from learned probability distributions, but their performance critically depends on the variable ordering used in the factorization via complexities of the resulting conditional distributions. We propose to learn the Markov random field describing the underlying data, and use the inferred graphical model structure to construct optimized variable orderings. We illustrate our approach on two-dimensional image-like models where a structure-aware ordering leads to restricted conditioning sets, thereby reducing model complexity. Numerical experiments on Ising models with discrete data demonstrate that graph-informed orderings yield higher-fidelity generated samples compared to naive variable orderings.
Paper Structure (16 sections, 15 equations, 7 figures)

This paper contains 16 sections, 15 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Related work.
  3. Problem and Methods
  4. Problem statement: autoregressive decompositions and variable ordering
  5. Conditional independence and parent sets
  6. Illustrative example.
  7. Reduced conditional distributions in the autoregressive decomposition
  8. Learning conditionals in the autoregressive decomposition.
  9. Selecting optimized ordering
  10. Learning of the Markov random field structure.
  11. Criterion for choosing optimized variable ordering.
  12. Results
  13. Exact training samples: $5 \times 5$ square lattice
  14. High-quality training samples: $10 \times 10$ ferro model
  15. Real data use case: D-Wave Dataset
  16. ...and 1 more sections

Figures (7)

  • Figure 1: An example graph for illustrating the Markov property and the reduction of conditioning sets.
  • Figure 2: Depiction of the three variable order traversals—sequential, checkerboard, and diagonal —on a $5\times 5$ lattice. Node shading indicates the relative selection order, from darker to lighter; identical shading denotes equal preference. Nodes without shading are selected after the shaded nodes and are not distinguished by the ordering. Node labels indicate one specific realization of the traversal.
  • Figure 3: Number of training data samples $M_l$ versus sampling error $\varepsilon$\ref{['eq:error']} for both ferro and spin glass Ising models with conditional order $O=6$. The number of generated samples $M_s = 10^5$. Error bars are one standard deviation over 20 independent training data sets.
  • Figure 4: Number of generated samples $M_s$ versus sampling error $\varepsilon$\ref{['eq:error']} for a ferromagnetic Ising model with different conditional orders $O\in \{2,4,6\}$, where the conditionals are learned from the ground-truth distribution $p$. Error bars indicate one standard deviation over 50 independently generated sample sets.
  • Figure 5: Number of training data samples $M_l$ versus sampling error $\varepsilon$\ref{['eq:error']} for ferro Ising model with model orders $O \in\{2,4\}$. The number of generated samples $M_s = 10^5$. Error bars indicate one standard deviation over 20 independent training data sets.
  • ...and 2 more figures

Theorems & Definitions (1)

  • Definition 2.1