Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bayesian Sparse Gaussian Mixture Model in High Dimensions

Dapeng Yao, Fangzheng Xie, Yanxun Xu

TL;DR

The proposed Bayesian sparse Gaussian mixture model does not require pre-specifying the number of clusters, which can be adaptively estimated via the Gibbs sampler and it is proved that the posterior contraction rate of the proposed Bayesian method is minimax optimal.

Abstract

We study the sparse high-dimensional Gaussian mixture model when the number of clusters is allowed to grow with the sample size. A minimax lower bound for parameter estimation is established, and we show that a constrained maximum likelihood estimator achieves the minimax lower bound. However, this optimization-based estimator is computationally intractable because the objective function is highly nonconvex and the feasible set involves discrete structures. To address the computational challenge, we propose a Bayesian approach to estimate high-dimensional Gaussian mixtures whose cluster centers exhibit sparsity using a continuous spike-and-slab prior. Posterior inference can be efficiently computed using an easy-to-implement Gibbs sampler. We further prove that the posterior contraction rate of the proposed Bayesian method is minimax optimal. The mis-clustering rate is obtained as a by-product using tools from matrix perturbation theory. The proposed Bayesian sparse Gaussian mixture model does not require pre-specifying the number of clusters, which can be adaptively estimated via the Gibbs sampler. The validity and usefulness of the proposed method is demonstrated through simulation studies and the analysis of a real-world single-cell RNA sequencing dataset.

Bayesian Sparse Gaussian Mixture Model in High Dimensions

TL;DR

The proposed Bayesian sparse Gaussian mixture model does not require pre-specifying the number of clusters, which can be adaptively estimated via the Gibbs sampler and it is proved that the posterior contraction rate of the proposed Bayesian method is minimax optimal.

Abstract

We study the sparse high-dimensional Gaussian mixture model when the number of clusters is allowed to grow with the sample size. A minimax lower bound for parameter estimation is established, and we show that a constrained maximum likelihood estimator achieves the minimax lower bound. However, this optimization-based estimator is computationally intractable because the objective function is highly nonconvex and the feasible set involves discrete structures. To address the computational challenge, we propose a Bayesian approach to estimate high-dimensional Gaussian mixtures whose cluster centers exhibit sparsity using a continuous spike-and-slab prior. Posterior inference can be efficiently computed using an easy-to-implement Gibbs sampler. We further prove that the posterior contraction rate of the proposed Bayesian method is minimax optimal. The mis-clustering rate is obtained as a by-product using tools from matrix perturbation theory. The proposed Bayesian sparse Gaussian mixture model does not require pre-specifying the number of clusters, which can be adaptively estimated via the Gibbs sampler. The validity and usefulness of the proposed method is demonstrated through simulation studies and the analysis of a real-world single-cell RNA sequencing dataset.
Paper Structure (22 sections, 10 theorems, 123 equations, 9 figures, 4 tables, 1 algorithm)

This paper contains 22 sections, 10 theorems, 123 equations, 9 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Model
  3. Gaussian mixture model and clustering
  4. Minimax lower bound
  5. Minimax upper bound and a constrained maximum likelihood estimator
  6. Bayesian sparse high-dimensional Gaussian mixture model
  7. Theoretical Properties
  8. Posterior contraction rate
  9. Mis-clustering error
  10. Simulation Studies
  11. Simulation setup
  12. Simulation results
  13. Single-cell Sequencing Data Analysis
  14. Discussion
  15. Proofs for Section \ref{['sec:model']}
  16. ...and 7 more sections

Key Result

Theorem 2.1

Let $\bm{Y} = (\bm{\mu}^*)(\bm{L}^*)^T + \bm{E}$ where each column of $\bm{E}$ is normal random vector with mean zero and covariance matrix $\bm{\Sigma}^*$. Assume Assumptions assump1-assump4 hold. If $s\leq p/4$, then there exists a constant $C>0$ such that for sufficiently large $n$, where $\mathop{\mathrm{\mathbb{E}}}\nolimits_*$ denotes the expected value with respect to $(\bm{\mu}^*,\bm{L}^*

Figures (9)

  • Figure 1: Clustering results of different methods compared to the true cluster memberships in Scenario I with $K^*=3$ and $s=6$ in a randomly selected simulation replicate. Data points are projected onto the subspace of the first two coordinates and different colors correspond to different estimated cluster memberships.
  • Figure 2: Numerical results of $\|\hat{\bm{\mu}}_1-\bm{\mu}^*_1\|_2$ of different methods in Scenario I with $K^*=3$ and $s=12$ across 100 simulation replicates.
  • Figure 3: Clustering results of different methods compared with the truth in one randomly selected simulation replicate of Scenario II. (a) Simulated truth. (b) Clustering result under the proposed Bayesian method. (c-f) Clustering results under SKM, PCA-KM, MClust, and CHIME when the number of clusters is fixed to be truth. Data points are projected onto the subspace of the first two coordinates and different colors correspond to different estimated cluster memberships.
  • Figure 4: Clustering results under different methods in one randomly selected replicate of Scenario III. Data points are projected onto the subspace of the first two coordinates and different colors correspond to different estimated cluster memberships.
  • Figure 5: Clustering results of scRNA-Seq data corresponding to different methods. Data points are embedded into two-dimensional subspace by tSNE embedding.
  • ...and 4 more figures

Theorems & Definitions (24)

  • Theorem 2.1
  • Remark 1
  • Theorem 2.2
  • Theorem 3.1
  • Remark 2
  • Corollary 3.1
  • Remark 3
  • Theorem 3.2
  • Remark 4
  • Remark 5
  • ...and 14 more