Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM-MemCluster: Empowering Large Language Models with Dynamic Memory for Text Clustering

Yuanjie Zhu, Liangwei Yang, Ke Xu, Weizhi Zhang, Zihe Song, Jindong Wang, Philip S. Yu

TL;DR

This work tackles unsupervised text clustering with large language models by overcoming LLM statelessness and the challenge of determining the number of clusters. It introduces LLM-MemCluster, which embeds a Dynamic Memory to maintain evolving cluster labels and uses a Dual-Prompt Strategy to actively control clustering granularity in a single-pass, end-to-end process. The approach achieves state-of-the-art results on six public benchmarks without model fine-tuning, significantly outperforming traditional embedding pipelines and prior LLM-based baselines. The findings suggest that architectural design choices—stateful memory and adaptive prompting—drive gains across diverse LLMs, enabling robust, tuning-free clustering in real-world settings.

Abstract

Large Language Models (LLMs) are reshaping unsupervised learning by offering an unprecedented ability to perform text clustering based on their deep semantic understanding. However, their direct application is fundamentally limited by a lack of stateful memory for iterative refinement and the difficulty of managing cluster granularity. As a result, existing methods often rely on complex pipelines with external modules, sacrificing a truly end-to-end approach. We introduce LLM-MemCluster, a novel framework that reconceptualizes clustering as a fully LLM-native task. It leverages a Dynamic Memory to instill state awareness and a Dual-Prompt Strategy to enable the model to reason about and determine the number of clusters. Evaluated on several benchmark datasets, our tuning-free framework significantly and consistently outperforms strong baselines. LLM-MemCluster presents an effective, interpretable, and truly end-to-end paradigm for LLM-based text clustering.

LLM-MemCluster: Empowering Large Language Models with Dynamic Memory for Text Clustering

TL;DR

This work tackles unsupervised text clustering with large language models by overcoming LLM statelessness and the challenge of determining the number of clusters. It introduces LLM-MemCluster, which embeds a Dynamic Memory to maintain evolving cluster labels and uses a Dual-Prompt Strategy to actively control clustering granularity in a single-pass, end-to-end process. The approach achieves state-of-the-art results on six public benchmarks without model fine-tuning, significantly outperforming traditional embedding pipelines and prior LLM-based baselines. The findings suggest that architectural design choices—stateful memory and adaptive prompting—drive gains across diverse LLMs, enabling robust, tuning-free clustering in real-world settings.

Abstract

Large Language Models (LLMs) are reshaping unsupervised learning by offering an unprecedented ability to perform text clustering based on their deep semantic understanding. However, their direct application is fundamentally limited by a lack of stateful memory for iterative refinement and the difficulty of managing cluster granularity. As a result, existing methods often rely on complex pipelines with external modules, sacrificing a truly end-to-end approach. We introduce LLM-MemCluster, a novel framework that reconceptualizes clustering as a fully LLM-native task. It leverages a Dynamic Memory to instill state awareness and a Dual-Prompt Strategy to enable the model to reason about and determine the number of clusters. Evaluated on several benchmark datasets, our tuning-free framework significantly and consistently outperforms strong baselines. LLM-MemCluster presents an effective, interpretable, and truly end-to-end paradigm for LLM-based text clustering.

Paper Structure

This paper contains 31 sections, 3 equations, 7 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Method
  3. Problem Formulation
  4. Framework Overview
  5. Stateful Clustering via Dynamic Memory
  6. Dual-Prompt Granularity Control
  7. Algorithmic Implementation and Complexity
  8. Experiments
  9. Experimental Setup
  10. Datasets
  11. Evaluation Metrics
  12. Baselines
  13. Main Results (RQ1)
  14. Ablation Study (RQ2)
  15. Hyperparameter Analysis (RQ3)
  16. ...and 16 more sections

Figures (7)

  • Figure 1: An overview of our proposed LLM-MemCluster framework. This figure illustrates the core iterative process, which is driven by a Dynamic Memory mechanism and the Dual-Prompt Strategy.
  • Figure 2: Comprehensive ablation study and adaptive clustering strategy comparison.
  • Figure 3: Hyperparameter sensitivity analysis of the prompt transition threshold, demonstrating robust and near-optimal performance across a wide range of values for representative datasets and on average.
  • Figure 4: The unified prompt template (system prompt).
  • Figure 5: The unified prompt template (user prompt, with Known labels from dynamic memory).
  • ...and 2 more figures