Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Fuzzy C-Means with Graph Embedding

Qiang Chen, Weizhong Yu, Feiping Nie, Xuelong Li

TL;DR

This work tackles automatic learning of the membership degree hyper-parameter in fuzzy C-Means (FCM) and the challenge of non-Gaussian cluster shapes. It introduces Adaptive Fuzzy C-Means with Graph Embedding (AFCM), which couples FCM with a generalized Gaussian mixture model and adds a graph-embedding regularizer to operate on learned manifolds, while a degenerate version reduces to a parameter-free FCM. The model employs an efficient alternating optimization with closed-form updates for the membership matrix, cluster centers, the adaptive hyper-parameter, and a projected data representation $\\tilde{X}$ obtained from a generalized eigenproblem. Empirical results on synthetic and real-world datasets demonstrate improved clustering performance, robustness to non-Gaussian structures, and consistent convergence, with ablation studies highlighting the benefits of one-stage clustering and manifold learning.

Abstract

Fuzzy clustering algorithms can be roughly categorized into two main groups: Fuzzy C-Means (FCM) based methods and mixture model based methods. However, for almost all existing FCM based methods, how to automatically selecting proper membership degree hyper-parameter values remains a challenging and unsolved problem. Mixture model based methods, while circumventing the difficulty of manually adjusting membership degree hyper-parameters inherent in FCM based methods, often have a preference for specific distributions, such as the Gaussian distribution. In this paper, we propose a novel FCM based clustering model that is capable of automatically learning an appropriate membership degree hyper-parameter value and handling data with non-Gaussian clusters. Moreover, by removing the graph embedding regularization, the proposed FCM model can degenerate into the simplified generalized Gaussian mixture model. Therefore, the proposed FCM model can be also seen as the generalized Gaussian mixture model with graph embedding. Extensive experiments are conducted on both synthetic and real-world datasets to demonstrate the effectiveness of the proposed model.

Adaptive Fuzzy C-Means with Graph Embedding

TL;DR

This work tackles automatic learning of the membership degree hyper-parameter in fuzzy C-Means (FCM) and the challenge of non-Gaussian cluster shapes. It introduces Adaptive Fuzzy C-Means with Graph Embedding (AFCM), which couples FCM with a generalized Gaussian mixture model and adds a graph-embedding regularizer to operate on learned manifolds, while a degenerate version reduces to a parameter-free FCM. The model employs an efficient alternating optimization with closed-form updates for the membership matrix, cluster centers, the adaptive hyper-parameter, and a projected data representation obtained from a generalized eigenproblem. Empirical results on synthetic and real-world datasets demonstrate improved clustering performance, robustness to non-Gaussian structures, and consistent convergence, with ablation studies highlighting the benefits of one-stage clustering and manifold learning.

Abstract

Fuzzy clustering algorithms can be roughly categorized into two main groups: Fuzzy C-Means (FCM) based methods and mixture model based methods. However, for almost all existing FCM based methods, how to automatically selecting proper membership degree hyper-parameter values remains a challenging and unsolved problem. Mixture model based methods, while circumventing the difficulty of manually adjusting membership degree hyper-parameters inherent in FCM based methods, often have a preference for specific distributions, such as the Gaussian distribution. In this paper, we propose a novel FCM based clustering model that is capable of automatically learning an appropriate membership degree hyper-parameter value and handling data with non-Gaussian clusters. Moreover, by removing the graph embedding regularization, the proposed FCM model can degenerate into the simplified generalized Gaussian mixture model. Therefore, the proposed FCM model can be also seen as the generalized Gaussian mixture model with graph embedding. Extensive experiments are conducted on both synthetic and real-world datasets to demonstrate the effectiveness of the proposed model.
Paper Structure (21 sections, 1 theorem, 45 equations, 3 figures, 3 tables, 2 algorithms)

This paper contains 21 sections, 1 theorem, 45 equations, 3 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. FCM
  4. Spectral Clustering
  5. Generalized Gaussian Mixture Model
  6. The Proposed Method
  7. Formulation
  8. Optimization
  9. Construction of the Affinity Matrix
  10. Computational Complexity Analysis
  11. Experiment
  12. Description of Datasets
  13. Experimental Settings
  14. Visualization Experiments on Toy Datasets
  15. Evaluation on Real-World Datasets
  16. ...and 6 more sections

Key Result

Proposition 1

The update equations of the maximum log-likelihood function in Eq. (mle) for the generalized Gaussian mixture model, which is optimized through the EM algorithm, are equivalent to the update equations of the following objective function. where $u_{ij}$ is the membership degree denoting the probability of sample $x_i$ being assigned to the $j$-th cluster, $\alpha = \{ {\alpha _1},{\alpha _2},..,{

Figures (3)

  • Figure 1: Illustration on graph based manifold learning. For data residing in a complex manifold, as depicted in (a), Euclidean distance may not accurately capture the true relationships between samples. However, by projecting the data according to the similarity graph into a simpler manifold, as illustrated in (b), the newly learned manifold facilitates easier clustering compared to the original manifold.
  • Figure 2: Clustering results on toy datasets. (a) and (b) depict the clustering outcomes of FCM. (c) and (d) show the clustering results of degenerate AFCM. (e)-(h) present the clustering results of AFCM, which projects data from the original manifold into a newly learned manifold and perform clustering in this new manifold rather than the original manifold.
  • Figure 3: Convergence and ACC curves for AFCM on eight real-world datasets.

Theorems & Definitions (2)

  • Remark 1
  • Proposition 1