Discriminative Ordering Through Ensemble Consensus

TL;DR

The paper tackles the challenge of evaluating and selecting among clustering models when cluster definitions vary and partial constraints exist. It introduces DISCOTEC, a discriminative ordering method that ranks base clusterings by the distance between each model's connectivity matrix $A^t$ and a consensus matrix $C$ derived from the ensemble, incorporating a binarised variant $Q$ to stabilize comparisons. A constraint-regularised extension uses $R=\frac{\sum D(A_{ab}^t\|1) + \sum D(A_{ab}^t\|0)}{n_{ML}+n_{CL}}$ to align rankings with must-link/cannot-link information, making the approach compatible with approximately observed constraints. Across synthetic and real datasets, the binarised DISCOTEC variant consistently achieves higher correlation with ground-truth partitions than traditional ensemble baselines, and constraint integration further improves ranking quality, enabling robust model selection in heterogeneous clustering scenarios.

Abstract

Evaluating the performance of clustering models is a challenging task where the outcome depends on the definition of what constitutes a cluster. Due to this design, current existing metrics rarely handle multiple clustering models with diverse cluster definitions, nor do they comply with the integration of constraints when available. In this work, we take inspiration from consensus clustering and assume that a set of clustering models is able to uncover hidden structures in the data. We propose to construct a discriminative ordering through ensemble clustering based on the distance between the connectivity of a clustering model and the consensus matrix. We first validate the proposed method with synthetic scenarios, highlighting that the proposed score ranks the models that best match the consensus first. We then show that this simple ranking score significantly outperforms other scoring methods when comparing sets of different clustering algorithms that are not restricted to a fixed number of clusters and is compatible with clustering constraints.

Figures (6)

• Figure 1: Evolution of the average Kendall's tau ($\uparrow$) correlation between ranking metrics and ARI with targets of synthetic partition when the label preservation rate $\rho_\text{max}$ increases.
• Figure 2: Evolution of Kendall's tau correlation ($\uparrow$) of the selected models per ranking method as the interpolation $\alpha$ varies between highly accurate models ($\alpha=0$) and non-accurate models ($\alpha=1$).
• Figure 3: Kendall's tau correlation ($\uparrow$) after adding must-link cannot-link constraints from an increasing number of random observations for the UCI datasets with mixed clustering models.
• Figure 4: Pearson correlation between ranking metrics and ARI with ground truth as the number of models in the ensemble increase. Each model has an accuracy bounded between 10% and $\rho_\text{max}$.
• Figure 5: Kendall's tau correlation between ranking metrics and ARI with ground truth as the number of models in the ensemble increase. Each model has an accuracy bounded between 10% and $\rho_\text{max}$.