Total Variation Distance Meets Probabilistic Inference
Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S. Meel, Dimitrios Myrisiotis, A. Pavan, N. V. Vinodchandran
TL;DR
This work reveals a structural bridge between total variation distance estimation and probabilistic inference on Bayes nets. By introducing a structure-preserving reduction and the novel notion of partial couplings, it shows how TV distance can be approximated through efficient probabilistic inference, yielding a fully polynomial randomized approximation scheme for Bayes nets with small treewidth. The results generalize beyond product distributions, demonstrate #P-hardness barriers for exact TV vs. uniform distance, and provide practical FPRASes with clear complexity bounds. The approach has potential implications for model comparison, sampler design, and broader probabilistic reasoning tasks in graphical models.
Abstract
In this paper, we establish a novel connection between total variation (TV) distance estimation and probabilistic inference. In particular, we present an efficient, structure-preserving reduction from relative approximation of TV distance to probabilistic inference over directed graphical models. This reduction leads to a fully polynomial randomized approximation scheme (FPRAS) for estimating TV distances between same-structure distributions over any class of Bayes nets for which there is an efficient probabilistic inference algorithm. In particular, it leads to an FPRAS for estimating TV distances between distributions that are defined over a common Bayes net of small treewidth. Prior to this work, such approximation schemes only existed for estimating TV distances between product distributions. Our approach employs a new notion of $partial$ couplings of high-dimensional distributions, which might be of independent interest.