Mixture-Models: a one-stop Python Library for Model-based Clustering using various Mixture Models

TL;DR

The paper presents Mixture-Models, a Python library for fitting Gaussian Mixture Models and variants in high-dimensional settings using automatic differentiation to enable gradient-based inference. It argues that gradient-based optimization (e.g., GD, Adam, Newton-CG) can outperform traditional EM in complex HD regimes, thanks to flexible reparametrizations and the ability to incorporate diverse models (MFA, PGMM, MClust) in a single framework. The work provides a modular design with a base MixtureModels class, thorough testing, and a broad simulation study comparing optimization strategies, concluding that the library offers a scalable, extensible, MIT-licensed tool for model-based clustering. It also outlines future directions, including PyTorch compatibility, to widen uptake among the deep learning and data science communities.

Abstract

\texttt{Mixture-Models} is an open-source Python library for fitting Gaussian Mixture Models (GMM) and their variants, such as Parsimonious GMMs, Mixture of Factor Analyzers, MClust models, Mixture of Student's t distributions, etc. It streamlines the implementation and analysis of these models using various first/second order optimization routines such as Gradient Descent and Newton-CG through automatic differentiation (AD) tools. This helps in extending these models to high-dimensional data, which is first of its kind among Python libraries. The library provides user-friendly model evaluation tools, such as BIC, AIC, and log-likelihood estimation. The source-code is licensed under MIT license and can be accessed at \url{https://github.com/kasakh/Mixture-Models}. The package is highly extensible, allowing users to incorporate new distributions and optimization techniques with ease. We conduct a large scale simulation to compare the performance of various gradient based approaches against Expectation Maximization on a wide range of settings and identify the corresponding best suited approach.

Figures (4)

• Figure 1: Workflow for using Mixture-Models library
• Figure 2: Comparing the performance of Gradient Descent (GD), Adam, Newton-CG and EM by varying $n$, $p$ and $K$
• Figure 3: Illustration of Subspace Clustering(taken from bouveyron2007high)
• Figure 4: Comparison of FA vs MFA(taken from bouveyron2014model)