Text Clustering as Classification with LLMs
Chen Huang, Guoxiu He
TL;DR
This work tackles unsupervised text clustering by removing the need for fine-tuned embeddings and traditional clustering algorithms. It reframes clustering as a two-stage classification problem using in-context learning in LLMs: first generating semantically meaningful candidate labels from data, then assigning samples to these labels to form clusters. Across five diverse datasets, the approach achieves comparable or superior clustering performance to embedding-based baselines while reducing computational overhead and enabling human-readable cluster explanations. The method demonstrates the potential of LLMs to simplify and enhance clustering with interpretable labels and provides public code for replication and extension.
Abstract
Text clustering serves as a fundamental technique for organizing and interpreting unstructured textual data, particularly in contexts where manual annotation is prohibitively costly. With the rapid advancement of Large Language Models (LLMs) and their demonstrated effectiveness across a broad spectrum of NLP tasks, an emerging body of research has begun to explore their potential in the domain of text clustering. However, existing LLM-based approaches still rely on fine-tuned embedding models and sophisticated similarity metrics, rendering them computationally intensive and necessitating domain-specific adaptation. To address these limitations, we propose a novel framework that reframes text clustering as a classification task by harnessing the in-context learning capabilities of LLMs. Our framework eliminates the need for fine-tuning embedding models or intricate clustering algorithms. It comprises two key steps: first, the LLM is prompted to generate a set of candidate labels based on the dataset and then merges semantically similar labels; second, it assigns the most appropriate label to each text sample. By leveraging the advanced natural language understanding and generalization capabilities of LLMs, the proposed approach enables effective clustering with minimal human intervention. Experimental results on diverse datasets demonstrate that our framework achieves comparable or superior performance to state-of-the-art embedding-based clustering techniques, while significantly reducing computational complexity and resource requirements. These findings underscore the transformative potential of LLMs in simplifying and enhancing text clustering tasks. We make our code available to the public for utilization at https://github.com/ECNU-Text-Computing/Text-Clustering-via-LLM. We also provide the supplementary Appendix within the repository.