Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Diagonal Gaussian Mixture Models and Higher Order Tensor Decompositions

Bingni Guo, Jiawang Nie, Zi Yang

TL;DR

This work addresses the problem of learning Gaussian mixtures with diagonal covariances by exploiting high-order moment tensors and incomplete symmetric tensor decompositions via generating polynomials. By moving beyond the third moment, it enables recovering more components and derives bounds on the maximum learnable rank $r_{\max}$ for given dimension $d$ and moment order $m$. The authors present a complete pipeline from moment estimation to parameter recovery, with stability guarantees: small estimation errors in moments imply proportional errors in the recovered means $\mu_i$, weights $\omega_i$, and covariances $\Sigma_i$. Numerical experiments on synthetic data demonstrate accurate recovery and favorable comparison to EM, validating the method's scalability to larger $r$ and higher $m$.

Abstract

This paper studies how to recover parameters in diagonal Gaussian mixture models using tensors. High-order moments of the Gaussian mixture model are estimated from samples. They form incomplete symmetric tensors generated by hidden parameters in the model. We propose to use generating polynomials to compute incomplete symmetric tensor approximations. The obtained decomposition is utilized to recover parameters in the model. We prove that our recovered parameters are accurate when the estimated moments are accurate. Using high-order moments enables our algorithm to learn Gaussian mixtures with more components. For a given model dimension and order, we provide an upper bound of the number of components in the Gaussian mixture model that our algorithm can compute.

Diagonal Gaussian Mixture Models and Higher Order Tensor Decompositions

TL;DR

This work addresses the problem of learning Gaussian mixtures with diagonal covariances by exploiting high-order moment tensors and incomplete symmetric tensor decompositions via generating polynomials. By moving beyond the third moment, it enables recovering more components and derives bounds on the maximum learnable rank for given dimension and moment order . The authors present a complete pipeline from moment estimation to parameter recovery, with stability guarantees: small estimation errors in moments imply proportional errors in the recovered means , weights , and covariances . Numerical experiments on synthetic data demonstrate accurate recovery and favorable comparison to EM, validating the method's scalability to larger and higher .

Abstract

This paper studies how to recover parameters in diagonal Gaussian mixture models using tensors. High-order moments of the Gaussian mixture model are estimated from samples. They form incomplete symmetric tensors generated by hidden parameters in the model. We propose to use generating polynomials to compute incomplete symmetric tensor approximations. The obtained decomposition is utilized to recover parameters in the model. We prove that our recovered parameters are accurate when the estimated moments are accurate. Using high-order moments enables our algorithm to learn Gaussian mixtures with more components. For a given model dimension and order, we provide an upper bound of the number of components in the Gaussian mixture model that our algorithm can compute.
Paper Structure (10 sections, 7 theorems, 122 equations, 5 tables)

This paper contains 10 sections, 7 theorems, 122 equations, 5 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Incomplete Symmetric Tensor Decompositions
  4. Incomplete Tensor Approximations and Error Analysis
  5. Learning General Diagonal Gaussian Mixture
  6. Numerical Experiments
  7. Incomplete tensor decomposition
  8. Incomplete tensor approximation
  9. Gaussian Mixture Approximation
  10. Conclusion

Key Result

Lemma 3.1

Suppose that $\binom{k}{p}\ge r$ and $\binom{n-k-1}{m-p-1}\ge r$. Let ${\mathcal{F}}_m$ be the tensor with the decomposition F-decomp. If vectors $\{[u_i]_{{\mathscr B}_0}\}_{i=1}^r$ and $\{[u_i]_{{\mathcal{O}}_\alpha}\}_{i=1}^r$ are both linearly independent, then the matrix $A[\alpha,{\mathcal{F}}

Theorems & Definitions (17)

  • Lemma 3.1
  • proof
  • Remark 3.2
  • Theorem 3.3
  • proof
  • Theorem 3.5
  • proof
  • Lemma 3.6
  • proof
  • Theorem 3.7
  • ...and 7 more