Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Statistical Inference for Fuzzy Clustering

Qiuyi Wu, Zihan Zhu, Anru R. Zhang

TL;DR

This work addresses the lack of principled statistical inference for fuzzy clustering in settings with imbalanced clusters and continuum phenotypes by introducing Weighted Fuzzy Clusters (WFCM). It formulates a probabilistic density f(x; V,w,m,σ) induced by the WFCM loss, and develops a blockwise MM algorithm with importance sampling for the normalizing constant, enabling likelihood-based estimation. The authors establish consistency and asymptotic normality of the MLE, and provide tools for inference, such as likelihood-ratio tests and bootstrap confidence regions, plus a weighted Xie-Beni index for model selection. In applications to scRNA-seq PBMC data and ADNI neuroimaging, WFCM yields stable uncertainty quantification and meaningful soft memberships, capturing both well-separated clusters and a disease-continuum in AD progression.

Abstract

Clustering is a central tool in biomedical research for discovering heterogeneous patient subpopulations, where group boundaries are often diffuse rather than sharply separated. Traditional methods produce hard partitions, whereas soft clustering methods such as fuzzy $c$-means (FCM) allow mixed memberships and better capture uncertainty and gradual transitions. Despite the widespread use of FCM, principled statistical inference for fuzzy clustering remains limited. We develop a new framework for weighted fuzzy $c$-means (WFCM) for settings with potential cluster size imbalance. Cluster-specific weights rebalance the classical FCM criterion so that smaller clusters are not overwhelmed by dominant groups, and the weighted objective induces a normalized density model with scale parameter $σ$ and fuzziness parameter $m$. Estimation is performed via a blockwise majorize--minimize (MM) procedure that alternates closed-form membership and centroid updates with likelihood-based updates of $(σ,\bw)$. The intractable normalizing constant is approximated by importance sampling using a data-adaptive Gaussian mixture proposal. We further provide likelihood ratio tests for comparing cluster centers and bootstrap-based confidence intervals. We establish consistency and asymptotic normality of the maximum likelihood estimator, validate the method through simulations, and illustrate it using single-cell RNA-seq and Alzheimer disease Neuroimaging Initiative (ADNI) data. These applications demonstrate stable uncertainty quantification and biologically meaningful soft memberships, ranging from well-separated cell populations under imbalance to a graded AD versus non-AD continuum consistent with disease progression.

Statistical Inference for Fuzzy Clustering

TL;DR

This work addresses the lack of principled statistical inference for fuzzy clustering in settings with imbalanced clusters and continuum phenotypes by introducing Weighted Fuzzy Clusters (WFCM). It formulates a probabilistic density f(x; V,w,m,σ) induced by the WFCM loss, and develops a blockwise MM algorithm with importance sampling for the normalizing constant, enabling likelihood-based estimation. The authors establish consistency and asymptotic normality of the MLE, and provide tools for inference, such as likelihood-ratio tests and bootstrap confidence regions, plus a weighted Xie-Beni index for model selection. In applications to scRNA-seq PBMC data and ADNI neuroimaging, WFCM yields stable uncertainty quantification and meaningful soft memberships, capturing both well-separated clusters and a disease-continuum in AD progression.

Abstract

Clustering is a central tool in biomedical research for discovering heterogeneous patient subpopulations, where group boundaries are often diffuse rather than sharply separated. Traditional methods produce hard partitions, whereas soft clustering methods such as fuzzy -means (FCM) allow mixed memberships and better capture uncertainty and gradual transitions. Despite the widespread use of FCM, principled statistical inference for fuzzy clustering remains limited. We develop a new framework for weighted fuzzy -means (WFCM) for settings with potential cluster size imbalance. Cluster-specific weights rebalance the classical FCM criterion so that smaller clusters are not overwhelmed by dominant groups, and the weighted objective induces a normalized density model with scale parameter and fuzziness parameter . Estimation is performed via a blockwise majorize--minimize (MM) procedure that alternates closed-form membership and centroid updates with likelihood-based updates of . The intractable normalizing constant is approximated by importance sampling using a data-adaptive Gaussian mixture proposal. We further provide likelihood ratio tests for comparing cluster centers and bootstrap-based confidence intervals. We establish consistency and asymptotic normality of the maximum likelihood estimator, validate the method through simulations, and illustrate it using single-cell RNA-seq and Alzheimer disease Neuroimaging Initiative (ADNI) data. These applications demonstrate stable uncertainty quantification and biologically meaningful soft memberships, ranging from well-separated cell populations under imbalance to a graded AD versus non-AD continuum consistent with disease progression.
Paper Structure (45 sections, 5 theorems, 82 equations, 12 figures, 2 tables, 2 algorithms)

This paper contains 45 sections, 5 theorems, 82 equations, 12 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Our Contributions
  3. Related Literature
  4. Organization
  5. Notation and Preliminaries on Classic Fuzzy $C$-Means
  6. A Statistical Framework for Weighted Fuzzy $C$-Means
  7. Weighted fuzzy $c$-means.
  8. A density induced by WFCM.
  9. Memberships, likelihood, and estimation
  10. Weighted Fuzzy $C$-Means: Algorithm and Statistical Inference
  11. Blockwise MM-Like Fuzzy Clustering Algorithm
  12. Initialization.
  13. Importance Sampling for the Normalizing Constant.
  14. Overall Algorithm
  15. Statistical Inference
  16. ...and 30 more sections

Key Result

Theorem 1

Let $\mathbf{x}_1,\dots,\mathbf{x}_n$ be i.i.d. draws from the true density $f_{\theta_0}$ with $\theta_0\in\operatorname{int}(\Theta)$. Define the MLE $\hat{\theta}_n =(\hat{\sigma}_n,\hat{\mathbf{w}}_n,\hat{\mathbf{V}}_n,\hat{m}_n) \in\mathop{\mathrm{arg max}}\limits_{\theta\in\Theta} \sum_{i=1}^{ up to a permutation of the $k$ component labels, namely, there exists a random permutation $\pi_n$

Figures (12)

  • Figure 1: Illustration of cluster imbalance in classic and weighted fuzzy $c$-means. Due to the imbalance in cluster sizes, the larger cluster exerts stronger influence on the optimization of classic FCM, causing centroid shifts toward denser regions and reducing membership accuracy for the smaller cluster in the left figure. While our proposed WFCM incorporates density-based weights to correct for sample imbalance and thus yields more balanced membership assignments and more accurate cluster centers for the minority group.
  • Figure 2: Induced density $f(\mathbf{x})$ in \ref{['eq:WFCM-density']} for different values of $\sigma^2$ and $m$ in two dimensions with two centroids (red crosses) and equal weights $w_1=w_2$. Smaller $m$ produces sharper, nearly hard-clustering modes; larger $m$ yields smoother, overlapping regions. Increasing $\sigma^2$ broadens the density and flattens the boundaries.
  • Figure 3: Finite-sample consistency of the MM fuzzy clustering estimator. Errors are shown as functions of sample size $n$ on a log–log scale.
  • Figure 4: Marginal QQ plots of whitened parameter estimates across $n \in \{100,200,500,1000,2000\}$. Each subplot corresponds to a selected coordinate of the parameter vector. The closer the empirical quantiles lie to the $45^\circ$ line, the stronger the agreement with the Gaussian benchmark.
  • Figure 5: Selection of the fuzziness parameter $m$ via approximate trainNLL. For each $m\in\{1.3,1.5,1.7,2.0,2.2,2.4,2.6\}$ we fit $(\sigma, \mathbf{V}, w)$ using the blockwise MM routine (50 iterations), and select the best $m$ using the negative log-likelihood.
  • ...and 7 more figures

Theorems & Definitions (6)

  • Remark 1: WFCM versus Gaussian mixture modeling
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • Lemma 5