Towards Calibrated Deep Clustering Network

TL;DR

This work tackles the problem of overconfidence in deep clustering by introducing Calibrated Deep Clustering (CDC), a dual-head network with a Calibration Head and a Clustering Head. The Calibration Head produces calibrated confidences using a region-aware loss with a mini-cluster target and a negative-entropy term, while the Clustering Head uses these confidences to dynamically select high-confidence pseudo-labels for self-training. A feature-prototype-based initialization, built on MoCo-v2 pretraining, ensures stable and fast convergence, and theoretical guarantees underpin the calibration strategy. Empirical results across six benchmarks show CDC achieving about a 5x reduction in expected calibration error ($ECE$) and substantial gains in clustering accuracy, with strong failure-rejection and OOD-detection capabilities. The approach enables trustworthy unsupervised clustering with reliable confidence estimates and practical impact on downstream decision-making tasks.

Abstract

Deep clustering has exhibited remarkable performance; however, the over confidence problem, i.e., the estimated confidence for a sample belonging to a particular cluster greatly exceeds its actual prediction accuracy, has been over looked in prior research. To tackle this critical issue, we pioneer the development of a calibrated deep clustering framework. Specifically, we propose a novel dual head (calibration head and clustering head) deep clustering model that can effectively calibrate the estimated confidence and the actual accuracy. The calibration head adjusts the overconfident predictions of the clustering head, generating prediction confidence that matches the model learning status. Then, the clustering head dynamically selects reliable high-confidence samples estimated by the calibration head for pseudo-label self-training. Additionally, we introduce an effective network initialization strategy that enhances both training speed and network robustness. The effectiveness of the proposed calibration approach and initialization strategy are both endorsed with solid theoretical guarantees. Extensive experiments demonstrate the proposed calibrated deep clustering model not only surpasses the state-of-the-art deep clustering methods by 5x on average in terms of expected calibration error, but also significantly outperforms them in terms of clustering accuracy. The code is available at https://github.com/ChengJianH/CDC.

Figures (11)

• Figure 1: Reliability diagrams of different methods on CIFAR-20. In the ideal case, the confidence of a model's output should be roughly equal to its accuracy, which means the confidence of the output is well-calibrated. However, the previous deep clustering models faced a severe overconfidence problem, i.e., the estimated confidence largely exceeds its real accuracy. We propose a calibrated deep clustering (CDC) model that enhances both confidence calibration and clustering performance.
• Figure 2: Illustration of the proposed CDC framework. The calibration head (CalHead) penalizes the overconfident predictions from the clustering head (CluHead). The clustering head, in turn, uses the calibrated confidence provided by the calibration head to select high-confidence samples for training. Note that the calibration head has the same structure as the clustering head.
• Figure 3: The failure rejection ability comparison on CIFAR-20. The second row shows the confidence distribution of correct and misclassified samples, demonstrating that our method has a stronger ability to separate failure predictions.
• Figure 4: The training process on CIFAR-10 and ImageNet-Dogs. CDC-Cal has (i) fewer training stages, (ii) better initialization strategy, and (iii) more stable performance improvement.
• Figure 5: The ACC and ECE are robust to varying K.

Theorems & Definitions (4)

• Theorem 1: Region-aware Penalty
• Theorem 2: Improve Calibration
• Proposition 1
• proof