Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A parameter-free clustering algorithm for missing datasets

Qi Li, Xianjun Zeng, Shuliang Wang, Wenhao Zhu, Shijie Ruan, Zhimeng Yuan

TL;DR

The paper tackles clustering on datasets with missing values without relying on imputations or tunable parameters. It introduces Single-Dimensional Clustering (SDC), which performs parameter-free, per-dimension decision-graph clustering and fuses results via partition intersection, augmented by gravity-based boundary contraction and a lightweight batch-density calculation to maintain efficiency. Key contributions include the first parameter-free approach for missing data, a novel partition-intersection fusion mechanism, a gravity-driven cluster-information enhancement, and an overall O((d+2)N log N) time complexity. Empirical results show SDC outperforms multiple baselines on NMI, ARI, and Purity across 13 missing-dataset benchmarks, with robust performance across varying missing rates. This work offers a scalable, parameter-free alternative for real-world incomplete data clustering.

Abstract

Missing datasets, in which some objects have missing values in certain dimensions, are prevalent in the Real-world. Existing clustering algorithms for missing datasets first impute the missing values and then perform clustering. However, both the imputation and clustering processes require input parameters. Too many input parameters inevitably increase the difficulty of obtaining accurate clustering results. Although some studies have shown that decision graphs can replace the input parameters of clustering algorithms, current decision graphs require equivalent dimensions among objects and are therefore not suitable for missing datasets. To this end, we propose a Single-Dimensional Clustering algorithm, i.e., SDC. SDC, which removes the imputation process and adapts the decision graph to the missing datasets by splitting dimension and partition intersection fusion, can obtain valid clustering results on the missing datasets without input parameters. Experiments demonstrate that, across three evaluation metrics, SDC outperforms baseline algorithms by at least 13.7%(NMI), 23.8%(ARI), and 8.1%(Purity).

A parameter-free clustering algorithm for missing datasets

TL;DR

The paper tackles clustering on datasets with missing values without relying on imputations or tunable parameters. It introduces Single-Dimensional Clustering (SDC), which performs parameter-free, per-dimension decision-graph clustering and fuses results via partition intersection, augmented by gravity-based boundary contraction and a lightweight batch-density calculation to maintain efficiency. Key contributions include the first parameter-free approach for missing data, a novel partition-intersection fusion mechanism, a gravity-driven cluster-information enhancement, and an overall O((d+2)N log N) time complexity. Empirical results show SDC outperforms multiple baselines on NMI, ARI, and Purity across 13 missing-dataset benchmarks, with robust performance across varying missing rates. This work offers a scalable, parameter-free alternative for real-world incomplete data clustering.

Abstract

Missing datasets, in which some objects have missing values in certain dimensions, are prevalent in the Real-world. Existing clustering algorithms for missing datasets first impute the missing values and then perform clustering. However, both the imputation and clustering processes require input parameters. Too many input parameters inevitably increase the difficulty of obtaining accurate clustering results. Although some studies have shown that decision graphs can replace the input parameters of clustering algorithms, current decision graphs require equivalent dimensions among objects and are therefore not suitable for missing datasets. To this end, we propose a Single-Dimensional Clustering algorithm, i.e., SDC. SDC, which removes the imputation process and adapts the decision graph to the missing datasets by splitting dimension and partition intersection fusion, can obtain valid clustering results on the missing datasets without input parameters. Experiments demonstrate that, across three evaluation metrics, SDC outperforms baseline algorithms by at least 13.7%(NMI), 23.8%(ARI), and 8.1%(Purity).
Paper Structure (18 sections, 10 theorems, 2 equations, 8 figures, 9 tables, 3 algorithms)

This paper contains 18 sections, 10 theorems, 2 equations, 8 figures, 9 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. The proposed method: SDC
  3. Clustering process
  4. Cluster-information enhancement process
  5. Lightweight Density Calculation Process
  6. Time Complexity Analysis
  7. Experiment
  8. Experimental Settings
  9. Ablation Experiments
  10. Comparison Experiments
  11. Related Work
  12. Conclusion
  13. The proof of theorems
  14. Time complexity analysis of the lightweight density calculation process
  15. Time complexity analysis of SDC
  16. ...and 3 more sections

Key Result

Theorem 2.1

Let $div(i)$ be $\{ {clu(i)}_{1},{clu(i)}_{2},\cdots,{clu(i)}_{S(i)}\}$ and $\{{clu}_{1},{clu}_{2},\cdots,{clu}_{S} \}$ be the true clusters in $X$. For $\forall x_{g} \in {clu(i)}_{k}$ and $\forall x_{r} \in {clu(i)}_{l}$, if ${clu(i)}_{k} \neq {clu(i)}_{l}$, then $\nexists{clu}_{p} \in \left\{ {cl

Figures (8)

  • Figure 1: The probabilities of achieving high and low accuracy for GAIN and MDIOT with different parameters.
  • Figure 2: An example of a density distribution decision graph
  • Figure 3: An example of the cluster-information enhancement process.
  • Figure 4: An example of lightweight density calculation.
  • Figure 5: The decision graphs comparison of SDC and SDC-NoEnhan on Overlap1 and Overlap2 datasets
  • ...and 3 more figures

Theorems & Definitions (22)

  • Theorem 2.1
  • Definition 2.2
  • Example 2.3
  • Theorem 2.4
  • Example 2.5
  • Definition 2.6
  • Example 2.7
  • Definition 2.8
  • Example 2.9
  • Definition 2.10
  • ...and 12 more