A new class of Markov random fields enabling lightweight sampling

TL;DR

The paper tackles the costly sampling of discrete MRFs (Potts/Ising) by connecting them to Gaussian Markov random fields through a unit-simplex mapping, yielding DGUMs that can be sampled directly via GMRF techniques. It introduces a multivariate GMRF $\mathbf{Z}$ with $(K-1)$ components, a softmax-based mapping $\pi_i^c$, and a discretization $\phi_{K,c}(\mathbf{Z})$ whose $c\to 0$ limit yields a discrete field $\mathbf{X}$. DGUM sampling leverages Fourier or spectral methods to achieve computational complexities of $\mathcal{O}((K-1)n\log n)$ or $\mathcal{O}((K-1)np)$, respectively, delivering about $35\times$ faster sampling and $37\times$ lower energy use than Gibbs in experiments while preserving key MRF properties like class balance and neighborhood-based similarity. This approach enables efficient image processing tasks and opens avenues for extensions to 3D lattices or graphs, as well as potential use in inverse problems despite a non-surjective back-mapping from $\mathbf{X}$ to $\mathbf{Z}$.

Abstract

This work addresses the problem of efficient sampling of Markov random fields (MRF). The sampling of Potts or Ising MRF is most often based on Gibbs sampling, and is thus computationally expensive. We consider in this work how to circumvent this bottleneck through a link with Gaussian Markov Random fields. The latter can be sampled in several cost-effective ways, and we introduce a mapping from real-valued GMRF to discrete-valued MRF. The resulting new class of MRF benefits from a few theoretical properties that validate the new model. Numerical results show the drastic performance gain in terms of computational efficiency, as we sample at least 35x faster than Gibbs sampling using at least 37x less energy, all the while exhibiting empirical properties close to classical MRFs.

Figures (11)

• Figure 1: Original intuition motivating this work: thresholding a GMRF realization (a) yields a discrete random field (b) that behaves similarly to a discrete MRF realization (c).
• Figure 2: Depiction of a realization of $\mathbf{Z}$ in the $K=3$ case, together with the mapping $\pi_k^c$, with $c=1$.
• Figure 3: Kernel density estimation obtained from the distribution of the triplet $(\pi_0^c(z_s),\pi_1^c(z_s),\pi_2^c(z_s))$ in the probability simplex whose vertices locates classes in $\Omega=\{\omega_0,\omega_1,\omega_2\}$, over all $s\in \mathcal{S}$ for a given realization $\mathbf{Z}=\mathbf{z}$. As $c \rightarrow \infty$, the distribution tends towards Dirac masses located at the vertices of the probability simplex.
• Figure 4: Illustration of the DGUM sampling for $K=3$ classes. The sampling is performed from $\mathbf{Z}=\mathbf{z}$ depicted in Fig. \ref{['fig:pi_i']}. (a)-(c) depict the GUM realization \ref{['eq:phi']}, and (d) depicts its limit DGUM realization $\mathbf{x}$\ref{['eq:dgum']}. (e) depicts the classes in $\mathbf{x}$ for the values of $\mathbf{z}$ in $\mathbb{R}^2$.
• Figure 5: Depiction of unbalanced (first line) and anisotropic (second line) GUM and DGUM distribution and realization, in the $K=3$ case. (a) and (f) depict the marginal locations in $\mathbb{R}^2$ of a realization $\mathbf{z}$, colored according to the class $\omega$, as in Fig. \ref{['fig:2d_gum']}. (b-d) and (g-i) depict the ternary distribution of $\pi$ (as in Fig. \ref{['fig:prob_simplex']}), showing in particular how these two situations enable class unbalance. (e) and (j) depict the corresponding DGUM realization.

Theorems & Definitions (16)

• Definition 1: Unit simplex
• Definition 2: Mapping with respect to unit simplex
• Remark 1
• Definition 3: Gaussian Unit-simplex Markov random field (GUM)
• Remark 2
• Definition 4: Discrete GUM
• Proposition 1: Markovianity of $\phi_{K,c}(\mathbf{Z})$
• proof
• Proposition 2: Markovianity of $\mathbf{X}$
• proof