Bayesian Inference for Discrete Markov Random Fields Through Coordinate Rescaling

TL;DR

This work tackles Bayesian inference for discrete MRFs, where the posterior is doubly intractable due to partition functions. It introduces coordinate-rescaling (CoRe) sampling to map from the efficient pseudo-posterior space to the target posterior, using a carefully chosen rescaling matrix A to capture the target’s covariance structure, with an adaptive variant AdaCoRe that updates A during burn-in. The authors compare CoRe against recalibration methods, empirical likelihood, and double Metropolis-Hastings approaches, demonstrating that CoRe achieves near-exact posterior variability and structure with substantially better efficiency than DMH variants, while retaining scalability similar to pseudo-likelihood. The results suggest CoRe as a robust, scalable solution for Bayesian inference in discrete MRFs and potential applicability to other composite-likelihood frameworks and graphical models.

Abstract

Discrete Markov random fields (MRFs) represent a class of undirected graphical models that capture complex conditional dependencies between discrete variables. Conducting exact posterior inference in these models is computationally challenging due to the intractable partition function, which depends on the model parameters and sums over all possible state configurations in the system. As a result, using the exact likelihood function is infeasible and existing methods, such as Double Metropolis-Hastings or pseudo-likelihood approximations, either scale poorly to large systems or underestimate the variability of the target posterior distribution. To address both computational burden and efficiency loss, we propose a new class of coordinate-rescaling sampling methods, which map the model parameters from the pseudo-likelihood space to the target posterior, preserving computational efficiency while improving posterior inference. Finally, in simulation studies, we compare the proposed method to existing approaches and illustrate that coordinate-rescaling sampling provides more accurate estimates of posterior variability, offering a scalable and robust solution for Bayesian inference in discrete MRFs.

Figures (11)

• Figure 1: (Left) Example of small-scale system consisting of six random variables. The grayscale intensity and edge thickness are proportional to the posterior absolute mode of the pairwise associations: stronger associations (in absolute value) correspond to darker and thicker edges. The parameter $\theta_{ij}$ quantifies the conditional dependence between the variables $X_i$ and $X_j$ given the remaining variables in the system. (Right) Posterior density of the pairwise association $\theta_{ij}$: the gray solid line represents the posterior results from the full likelihood (Exact posterior), the orange dashed line the posterior based on pseudo-likelihood function (Pseudo posterior) and the dashed purple line the posterior based on the Double Metropolis-Hastings algorithm (DMH posterior).
• Figure 2: Contour lines representing the posterior distribution of two pairwise association parameters, $\theta_{ij}$ and $\theta_{im}$. (Left) The pseudo posterior distribution (orange dashed contours) and the exact posterior (gray solid contours). (Center) The CoRe posterior distribution (gold dashed contours) and the exact posterior (gray solid contours). (Right) The DMH posterior distribution (purple dashed contours) and the exact posterior (gray solid contours).
• Figure 3: Posterior density of the pairwise association $\theta_{ij}$: the gray solid line represents the posterior results from the full likelihood (Exact posterior), the orange dashed line the posterior based on pseudo-likelihood function (Pseudo posterior), the dashed purple line the posterior based on the Double Metropolis-Hastings algorithm (DMH posterior) and the dotdashed yellow line the posterior based on the CoRe sampling (CoRe posterior).
• Figure 4: Examples of three graph structures for six nodes.
• Figure 5: Overlapping proportion index $\eta$ between exact and corrected posterior distributions for the interaction parameters in a network with "full" structure. The horizontal dashed gray line marks a reference level at 0.8.