Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Approximating Univariate Factored Distributions via Message-Passing Algorithms

Zilu Zhao, Dirk Slock

TL;DR

The paper tackles estimating statistics of univariate factored PDFs formed by products of GMM factors, where direct computation is intractable due to exponential growth in components. It introduces Variable Duplication BP (VDBP), which reformulates the problem as a multivariate linear-Gaussian model and applies GaBP to obtain marginal posteriors corresponding to the original univariate PDF. To handle non-integrable beliefs that arise in EP, two schemes are proposed: Persistent EP (passively skipping problematic updates) and Analytic Continuation EP (ACEP), which actively constrain the projection domain using natural-parameter Gaussian representations. Simulation results demonstrate that VDBP and the proposed EP variants outperform clipping EP in estimating mean and variance, especially for GMM-factorized PDFs, with ACEP achieving strong accuracy and robustness. The methods offer practical, scalable tools for bilinear estimation contexts and other setups where univariate factored distributions arise from products of mixture models.

Abstract

Gaussian Mixture Models (GMMs) commonly arise in communication systems, particularly in bilinear joint estimation and detection problems. Although the product of GMMs is still a GMM, as the number of factors increases, the number of components in the resulting product GMM grows exponentially. To obtain a tractable approximation for a univariate factored probability density function (PDF), such as a product of GMMs, we investigate iterative message-passing algorithms. Based on Belief Propagation (BP), we propose a Variable Duplication and Gaussian Belief Propagation (VDBP)-based algorithm. The key idea of VDBP is to construct a multivariate measurement model whose marginal posterior is equal to the given univariate factored PDF. We then apply Gaussian BP (GaBP) to transform the global inference problem into local ones. Expectation propagation (EP) is another branch of message passing algorithms. In addition to converting the global approximation problem into local ones, it features a projection operation that ensures the intermediate functions (messages) belong to a desired family. Due to this projection, EP can be used to approximate the factored PDF directly. However, even if every factor is integrable, the division operation in EP may still cause the algorithm to fail when the mean and variance of a non-integrable belief are required. Therefore, this paper proposes two methods that combine EP with our previously proposed techniques for handling non-integrable beliefs to approximate univariate factored distributions.

Approximating Univariate Factored Distributions via Message-Passing Algorithms

TL;DR

The paper tackles estimating statistics of univariate factored PDFs formed by products of GMM factors, where direct computation is intractable due to exponential growth in components. It introduces Variable Duplication BP (VDBP), which reformulates the problem as a multivariate linear-Gaussian model and applies GaBP to obtain marginal posteriors corresponding to the original univariate PDF. To handle non-integrable beliefs that arise in EP, two schemes are proposed: Persistent EP (passively skipping problematic updates) and Analytic Continuation EP (ACEP), which actively constrain the projection domain using natural-parameter Gaussian representations. Simulation results demonstrate that VDBP and the proposed EP variants outperform clipping EP in estimating mean and variance, especially for GMM-factorized PDFs, with ACEP achieving strong accuracy and robustness. The methods offer practical, scalable tools for bilinear estimation contexts and other setups where univariate factored distributions arise from products of mixture models.

Abstract

Gaussian Mixture Models (GMMs) commonly arise in communication systems, particularly in bilinear joint estimation and detection problems. Although the product of GMMs is still a GMM, as the number of factors increases, the number of components in the resulting product GMM grows exponentially. To obtain a tractable approximation for a univariate factored probability density function (PDF), such as a product of GMMs, we investigate iterative message-passing algorithms. Based on Belief Propagation (BP), we propose a Variable Duplication and Gaussian Belief Propagation (VDBP)-based algorithm. The key idea of VDBP is to construct a multivariate measurement model whose marginal posterior is equal to the given univariate factored PDF. We then apply Gaussian BP (GaBP) to transform the global inference problem into local ones. Expectation propagation (EP) is another branch of message passing algorithms. In addition to converting the global approximation problem into local ones, it features a projection operation that ensures the intermediate functions (messages) belong to a desired family. Due to this projection, EP can be used to approximate the factored PDF directly. However, even if every factor is integrable, the division operation in EP may still cause the algorithm to fail when the mean and variance of a non-integrable belief are required. Therefore, this paper proposes two methods that combine EP with our previously proposed techniques for handling non-integrable beliefs to approximate univariate factored distributions.
Paper Structure (31 sections, 4 theorems, 102 equations, 5 figures, 3 algorithms)

This paper contains 31 sections, 4 theorems, 102 equations, 5 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Prior Works
  3. Belief Propagation
  4. Expectation Propagation
  5. Main Contribution
  6. Introduction to BP
  7. Belief Propagation Rules
  8. Factor-to-Variable Message
  9. Variable-to-Factor Message
  10. Approximation Results of BP
  11. Introduction to EP
  12. Expectation Propagation Rules
  13. Belief at Factor Node
  14. Marginalized Factor Belief
  15. Project to Desired (Gaussian) Family
  16. ...and 16 more sections

Key Result

Lemma 1

$\forall {\mathbf A}\in \mathbb{R}^{(N-1)\times N}$ and $\forall n\in\{1, \dots, N\}$, let ${\overline{{\mathbf A}}}_n$ denote the matrix obtained by removing the $n$-th column of ${\mathbf A}$. If $\hbox{rank}({\mathbf A})=N-1$ and ${\mathbf A}\cdot {\boldsymbol 1}={\boldsymbol 0}$, then $\hbox{ran

Figures (5)

  • Figure 1: Factor Graph for Joint pdf $p(\theta_1, \theta_2, \theta_3)\propto f_{\alpha}(\theta_1, \theta_2)f_{\beta}(\theta_2, \theta_3)$
  • Figure 2: Factor Graph for VDBP
  • Figure 3: Factor Graph for EP
  • Figure 4: NSE CDF of mean estimates.
  • Figure 5: NSE CDF of variance estimates.

Theorems & Definitions (8)

  • Lemma 1
  • proof
  • Theorem 1
  • proof
  • Proposition 1
  • proof
  • Proposition 2
  • proof