Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Dirichlet Process mixture model with unknown concentration parameter and variance: Scaling high dimensional clustering via collapsed variational inference

Annesh Pal, Aguirre Mimoun, Rodolphe Thiébaut, Boris P. Hejblum

TL;DR

The paper tackles scalable clustering with Dirichlet process mixture models by introducing collapsed variational inference that jointly learns the DP concentration parameter $\alpha$ and the base distribution covariance $G_0$. It demonstrates that cluster-specific, sparsity-inducing covariance priors (Sparse DPMM) yield superior performance in high dimensions, with faster convergence than MCMC methods and robust clustering in real gene-expression data. Key contributions include a practical CVI framework for adaptive DPMM, extensive hyperparameter analyses, and an open-source R implementation (vimixr) validated on Gaussian, Negative Binomial, and leukemia datasets. The work has practical implications for high-dimensional clustering in genomics and related fields, offering a principled and scalable alternative to traditional finite mixtures and MCMC approaches.

Abstract

We propose a novel method that performs adaptive clustering with DPMM using collapsed VI, while incorporating weakly-informative priors for DP concentration parameter alpha and base distribution G0. We illustrate the importance of G0 covariance structure and prior choice by considering different parameterisations of the data covariance matrix. On high-dimensional Gaussian simulations, our model demonstrates substantially faster convergence than a state-of-the-art MCMC splice sampler. We further evaluate performances on Negative Binomial simulations and conduct sensitivity analyses to assess robustness on realistic data conditions. Application to a publicly available leukemia transcriptomic data set comprising 72 samples and 2,194 gene expression successfully recovers every known sub-type, all while identifying additional gene expression-based sub-clusters with meaningful biological interpretation.

Adaptive Dirichlet Process mixture model with unknown concentration parameter and variance: Scaling high dimensional clustering via collapsed variational inference

TL;DR

The paper tackles scalable clustering with Dirichlet process mixture models by introducing collapsed variational inference that jointly learns the DP concentration parameter and the base distribution covariance . It demonstrates that cluster-specific, sparsity-inducing covariance priors (Sparse DPMM) yield superior performance in high dimensions, with faster convergence than MCMC methods and robust clustering in real gene-expression data. Key contributions include a practical CVI framework for adaptive DPMM, extensive hyperparameter analyses, and an open-source R implementation (vimixr) validated on Gaussian, Negative Binomial, and leukemia datasets. The work has practical implications for high-dimensional clustering in genomics and related fields, offering a principled and scalable alternative to traditional finite mixtures and MCMC approaches.

Abstract

We propose a novel method that performs adaptive clustering with DPMM using collapsed VI, while incorporating weakly-informative priors for DP concentration parameter alpha and base distribution G0. We illustrate the importance of G0 covariance structure and prior choice by considering different parameterisations of the data covariance matrix. On high-dimensional Gaussian simulations, our model demonstrates substantially faster convergence than a state-of-the-art MCMC splice sampler. We further evaluate performances on Negative Binomial simulations and conduct sensitivity analyses to assess robustness on realistic data conditions. Application to a publicly available leukemia transcriptomic data set comprising 72 samples and 2,194 gene expression successfully recovers every known sub-type, all while identifying additional gene expression-based sub-clusters with meaningful biological interpretation.
Paper Structure (21 sections, 11 equations, 7 figures, 1 table)

This paper contains 21 sections, 11 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Problem set-up
  4. Dirichlet process mixture model
  5. Choice of $G_0$
  6. Posterior estimation using collapsed VI
  7. Latent allocations $\{z_n\}$
  8. Distributional parameters $\{\mu_k, \Sigma_k\}$
  9. Concentration parameter $\alpha$
  10. Implementation
  11. Results
  12. Effect of $G_0$ Covariance Structure and Prior Choice
  13. Effect of hyper-parameters of cluster distributions $\{\mu_k, \Sigma_k \}$
  14. Computational scalability
  15. Cancer sub-type estimation
  16. ...and 6 more sections

Figures (7)

  • Figure 1: $K_{post}$ and ARI scores for different model choice
  • Figure 2: ARI for different cluster-specific models when $K_{true}$ is varying
  • Figure 3: Sensitivity analysis of Sparse DPMM over the hyper-parameters \ref{['fig:3']}(a) $a_0=b_0$ and \ref{['fig:3']}(b) $k_0$ for Negative Binomial simulations with $K_{true}=3$
  • Figure 4: PCA projection on the first $2$ principal components for (a) labelled Leukemia sub-types based on $2194$ genes and (b) Sparse DPMM cluster estimates obtained using strong hyper-parameters
  • Figure 5: PCA projection on the first $2$ principal components for Sparse DPMM cluster estimates obtained using weaker hyper-prarameters
  • ...and 2 more figures