Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimized Algorithms for Text Clustering with LLM-Generated Constraints

Chaoqi Jia, Weihong Wu, Longkun Guo, Zhigang Lu, Chao Chen, Kok-Leong Ong

TL;DR

This work tackles short text clustering under must-link and cannot-link constraints, introducing LLM-generated constraint sets and a penalty-augmented local search to tolerate noisy outputs. It replaces traditional pairwise constraints with set-based ML/CL constraints and designs a two-stage workflow: constraint generation guided by LLMs and a constrained clustering algorithm that uses hard ML for initialization and soft ML/CL with penalties for refinement. The approach emphasizes query efficiency and robustness, employing coresets-based ML candidate sets, distance-based CL sampling, and an auxiliary bipartite graph with a minimum-weight matching to handle CL constraints. Empirical results on five real-world datasets show clustering accuracy on par with state-of-the-art methods while reducing LLM queries by more than 20x, highlighting practical efficiency and resilience to imperfect constraints.

Abstract

Clustering is a fundamental tool that has garnered significant interest across a wide range of applications including text analysis. To improve clustering accuracy, many researchers have incorporated background knowledge, typically in the form of must-link and cannot-link constraints, to guide the clustering process. With the recent advent of large language models (LLMs), there is growing interest in improving clustering quality through LLM-based automatic constraint generation. In this paper, we propose a novel constraint-generation approach that reduces resource consumption by generating constraint sets rather than using traditional pairwise constraints. This approach improves both query efficiency and constraint accuracy compared to state-of-the-art methods. We further introduce a constrained clustering algorithm tailored to the characteristics of LLM-generated constraints. Our method incorporates a confidence threshold and a penalty mechanism to address potentially inaccurate constraints. We evaluate our approach on five text datasets, considering both the cost of constraint generation and the overall clustering performance. The results show that our method achieves clustering accuracy comparable to the state-of-the-art algorithms while reducing the number of LLM queries by more than 20 times.

Optimized Algorithms for Text Clustering with LLM-Generated Constraints

TL;DR

This work tackles short text clustering under must-link and cannot-link constraints, introducing LLM-generated constraint sets and a penalty-augmented local search to tolerate noisy outputs. It replaces traditional pairwise constraints with set-based ML/CL constraints and designs a two-stage workflow: constraint generation guided by LLMs and a constrained clustering algorithm that uses hard ML for initialization and soft ML/CL with penalties for refinement. The approach emphasizes query efficiency and robustness, employing coresets-based ML candidate sets, distance-based CL sampling, and an auxiliary bipartite graph with a minimum-weight matching to handle CL constraints. Empirical results on five real-world datasets show clustering accuracy on par with state-of-the-art methods while reducing LLM queries by more than 20x, highlighting practical efficiency and resilience to imperfect constraints.

Abstract

Clustering is a fundamental tool that has garnered significant interest across a wide range of applications including text analysis. To improve clustering accuracy, many researchers have incorporated background knowledge, typically in the form of must-link and cannot-link constraints, to guide the clustering process. With the recent advent of large language models (LLMs), there is growing interest in improving clustering quality through LLM-based automatic constraint generation. In this paper, we propose a novel constraint-generation approach that reduces resource consumption by generating constraint sets rather than using traditional pairwise constraints. This approach improves both query efficiency and constraint accuracy compared to state-of-the-art methods. We further introduce a constrained clustering algorithm tailored to the characteristics of LLM-generated constraints. Our method incorporates a confidence threshold and a penalty mechanism to address potentially inaccurate constraints. We evaluate our approach on five text datasets, considering both the cost of constraint generation and the overall clustering performance. The results show that our method achieves clustering accuracy comparable to the state-of-the-art algorithms while reducing the number of LLM queries by more than 20 times.
Paper Structure (23 sections, 3 theorems, 2 equations, 2 figures, 2 tables, 3 algorithms)

This paper contains 23 sections, 3 theorems, 2 equations, 2 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Problem Formulation
  4. Methodology
  5. Algorithmic Framework
  6. Constraint Generation with LLM
  7. Must-Links Constraint Set.
  8. Cannot-Links Constraint Set.
  9. Clustering with LLM-generated Constraints
  10. Cluster Initialization
  11. ML Clustering with Penalty
  12. CL Clustering with Penalty
  13. Auxiliary Bipartite Graph
  14. Minimum Weight Perfect Matching
  15. Clustering with Both ML and CL Constraints
  16. ...and 8 more sections

Key Result

Lemma 1

The running time of Alg. alg:ML_penalty_clustering_confi is $O(nk^2)$.

Figures (2)

  • Figure 1: Comparison of average clustering results on datasets across different constrained instance ratios.
  • Figure 2: Comparison of clustering accuracy (ACC) across different large language models and algorithms with a $30\%$ constrained-instance ratio. Note that FSC in viswanathan2024large uses its own constraint‑generation algorithm and applies PCK for clustering.

Theorems & Definitions (7)

  • Lemma 1
  • proof
  • Definition 1
  • Definition 2
  • Lemma 2
  • proof
  • Lemma 3