Scalable Parameter-Light Spectral Method for Clustering Short Text Embeddings with a Cohesion-Based Evaluation Metric

TL;DR

This work tackles clustering short-text embeddings without predefined cluster counts by introducing a scalable spectral estimator based on the normalized Laplacian eigenspectrum of cosine-similarity graphs and an adaptive sampling strategy. It also introduces the Cohesion Ratio, an intrinsic evaluation metric that quantifies intra-cluster cohesion relative to the global similarity background and links to information-theoretic notions. The method, evaluated on six datasets across multiple embedding models, shows that using the spectral estimator to guide K-Means or HAC yields superior performance compared to popular parameter-light baselines such as HDBSCAN, OPTICS, and Leiden. The work provides practical, scalable tools for unsupervised organization and evaluation of short-text data and offers avenues for further refinement and integration into broader clustering frameworks.

Abstract

Clustering short text embeddings is a foundational task in natural language processing, yet remains challenging due to the need to specify the number of clusters in advance. We introduce a scalable spectral method that estimates the number of clusters directly from the structure of the Laplacian eigenspectrum, constructed using cosine similarities and guided by an adaptive sampling strategy. This sampling approach enables our estimator to efficiently scale to large datasets without sacrificing reliability. To support intrinsic evaluation of cluster quality without ground-truth labels, we propose the Cohesion Ratio, a simple and interpretable evaluation metric that quantifies how much intra-cluster similarity exceeds the global similarity background. It has an information-theoretic motivation inspired by mutual information, and in our experiments it correlates closely with extrinsic measures such as normalized mutual information and homogeneity. Extensive experiments on six short-text datasets and four modern embedding models show that standard algorithms like K-Means and HAC, when guided by our estimator, significantly outperform popular parameter-light methods such as HDBSCAN, OPTICS, and Leiden. These results demonstrate the practical value of our spectral estimator and Cohesion Ratio for unsupervised organization and evaluation of short text data. Implementation of our estimator of k and Cohesion Ratio, along with code for reproducing the experiments, is available at https://anonymous.4open.science/r/towards_clustering-0C2E.

Figures (6)

• Figure 1: First half of Laplacian eigenvalue spectrum and normalized spectral differences ($\delta_i$) for the BLESS dataset. The detected number of clusters $\hat{k}$ is shown as a vertical dashed line. The detection window (light gray) marks the region used for local averaging before the detected jump. The dotted line represents $\delta_i$ values, and the horizontal dashed line indicates the adaptive threshold computed from the mean and variance of spectral differences. A significant over-threshold jump at $\hat{k}$ signals the point where spectral flattening begins, suggesting the cut-off for meaningful clustering.
• Figure 2: Impact of window size $w$ on mean relative error (solid line, left axis) and average fraction of invalid predictions (dashed line, right axis).
• Figure 3: Spearman correlations between intrinsic and extrinsic clustering metrics. Higher values indicate stronger agreement with extrinsic clustering scores.
• Figure 4: Predicted number of clusters (log scale) across stratified datasets for different clustering algorithms. Each box shows the distribution over multiple runs. Black diamonds indicate the true number of underlying classes. Our method tends to produce cluster counts closest to ground truth across most datasets.
• Figure 5: Heatmaps of cosine similarity matrices for BLESS dataset and two stratified samples of DBpedia and Reddit datasets. Each pair shows the (left) raw cosine similarities and (right) z-score normalized and rectified versions. The first row corresponds to embeddings from the multilingual-e5-large-instruct model (smallest model from our experiments), while the second row uses the Qwen3-Embedding-8B model (largest model from our experiments). Normalization emphasizes strong intra-cluster similarities while suppressing noise, resulting in clearer block-like structures aligned with true class boundaries in some cases.