Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Probabilistic Inference in the Era of Tensor Networks and Differential Programming

Martin Roa-Villescas, Xuanzhao Gao, Sander Stuijk, Henk Corporaal, Jin-Guo Liu

TL;DR

The integration of quantum circuit simulation, quantum many-body physics, and statistical physics technologies with a series of algorithms introduced in this study significantly improves the performance efficiency of existing methods for solving probabilistic inference tasks.

Abstract

Probabilistic inference is a fundamental task in modern machine learning. Recent advances in tensor network (TN) contraction algorithms have enabled the development of better exact inference methods. However, many common inference tasks in probabilistic graphical models (PGMs) still lack corresponding TN-based adaptations. In this work, we advance the connection between PGMs and TNs by formulating and implementing tensor-based solutions for the following inference tasks: (i) computing the partition function, (ii) computing the marginal probability of sets of variables in the model, (iii) determining the most likely assignment to a set of variables, and (iv) the same as (iii) but after having marginalized a different set of variables. We also present a generalized method for generating samples from a learned probability distribution. Our work is motivated by recent technical advances in the fields of quantum circuit simulation, quantum many-body physics, and statistical physics. Through an experimental evaluation, we demonstrate that the integration of these quantum technologies with a series of algorithms introduced in this study significantly improves the effectiveness of existing methods for solving probabilistic inference tasks.

Probabilistic Inference in the Era of Tensor Networks and Differential Programming

TL;DR

The integration of quantum circuit simulation, quantum many-body physics, and statistical physics technologies with a series of algorithms introduced in this study significantly improves the performance efficiency of existing methods for solving probabilistic inference tasks.

Abstract

Probabilistic inference is a fundamental task in modern machine learning. Recent advances in tensor network (TN) contraction algorithms have enabled the development of better exact inference methods. However, many common inference tasks in probabilistic graphical models (PGMs) still lack corresponding TN-based adaptations. In this work, we advance the connection between PGMs and TNs by formulating and implementing tensor-based solutions for the following inference tasks: (i) computing the partition function, (ii) computing the marginal probability of sets of variables in the model, (iii) determining the most likely assignment to a set of variables, and (iv) the same as (iii) but after having marginalized a different set of variables. We also present a generalized method for generating samples from a learned probability distribution. Our work is motivated by recent technical advances in the fields of quantum circuit simulation, quantum many-body physics, and statistical physics. Through an experimental evaluation, we demonstrate that the integration of these quantum technologies with a series of algorithms introduced in this study significantly improves the effectiveness of existing methods for solving probabilistic inference tasks.
Paper Structure (13 sections, 3 theorems, 28 equations, 4 figures, 1 table)

This paper contains 13 sections, 3 theorems, 28 equations, 4 figures, 1 table.

Table of Contents

  1. Tensor networks
  2. Tensor networks for probabilistic modeling
  3. Partition Function (PR)
  4. Marginal Probability (MAR)
  5. Most Probable Explanation (MPE)
  6. Tropical tensor networks
  7. The most probable configuration
  8. Back-propagation in tropical tensor networks
  9. Maximum Marginal a Posteriori (MMAP)
  10. Generative modeling
  11. Performance benchmarks
  12. Conclusions
  13. Backward rule for tensor contraction

Key Result

Theorem 2.1

Let $(\Lambda, \mathcal{T}, \varnothing)$ be a tensor network with scalar output. The gradient of the tensor network contraction with respect to $T_V \in \mathcal{T}$ is That is, the gradient corresponds to the contraction of the tensor network with the tensor $T_V$ removed and the output label set to $V$.

Figures (4)

  • Figure 1: The contraction of a tensor network with three parts, $A_X$, $B_Y$, and $R_Z$.
  • Figure 2: Probabilistic interpretation of popular tensor networks. Dashed arrows denote the variable elimination order. Red edges correspond to the variables of interest. The set of gray tensors is the complex conjugate of the black tensors.
  • Figure 3: Runtime speedup achieved by our tensor-based library, TensorInference.jl, across four different probabilistic inference tasks, relative to Merlinmarinescu2022merlin, libDAImooij2010libdai and JunctionTrees.jlroa2022partial. The experiments were conducted on a CPU using the UAI 2014 inference competition benchmark problems.
  • Figure 4: TensorInference.jl's runtime speedup on a GPU for the MMAP task, relative to CPU performance, benchmarked on the UAI 2014 inference competition problems.

Theorems & Definitions (9)

  • Definition 1.1: Tensor
  • Definition 1.2: Tensor Network liu2022computingcirac2021matrixorus2014practical
  • Theorem 2.1: Tensor network differentiation
  • Corollary 2.1
  • proof
  • Definition 2.1: Tropical Tensor Network
  • Theorem 2.2
  • proof
  • proof