Stein's method for marginals on large graphical models
Tiangang Cui, Shuigen Liu, Xin T. Tong
TL;DR
The paper addresses the challenge of accurately approximating high-dimensional distributions by exploiting locality structures to obtain dimension-free guarantees for low-dimensional marginals. It introduces δ-locality, connects it to sparse graphical models and diagonal dominance, and uses Stein's method to derive a marginal Otto-Villani inequality that bounds marginals in terms of coordinate-wise score differences. The authors develop two localized methodologies—Localized LIS (LLIS) and Localized Score Matching (LSM)—and prove that these approaches achieve reduced sample complexity and scalable, parallelizable computations while preserving accuracy for marginals. The results offer a principled framework for scaling distribution approximation and sampling in large graphical models with locality, with potential impact on Bayesian inference and generative modeling in high dimensions.
Abstract
Many spatial models exhibit locality structures that effectively reduce their intrinsic dimensionality, enabling efficient approximation and sampling of high-dimensional distributions. However, existing approximation techniques mainly focus on joint distributions, and do not guarantee accuracy for low-dimensional marginals. By leveraging the locality structures, we establish a dimension independent uniform error bound for the marginals of approximate distributions. Inspired by the Stein's method, we introduce a novel $δ$-locality condition that quantifies the locality in distributions, and link it to the structural assumptions such as the sparse graphical models. The theoretical guarantee motivates the localization of existing sampling methods, as we illustrate through the localized likelihood-informed subspace method and localized score matching. We show that by leveraging the locality structure, these methods greatly reduce the sample complexity and computational cost via localized and parallel implementations.