Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM-Guided Lifecycle-Aware Clustering of Multi-Turn Customer Support Conversations

Priyaranjan Pattnayak, Sanchari Chowdhuri, Amit Agarwal, Hitesh Laxmichand Patel

TL;DR

This work addresses the challenge of clustering evolving, multi-turn customer support conversations across multiple cloud services. It introduces a lifecycle-aware framework that uses LLMs to segment chats into service-specific concerns, then applies incremental clustering within service groups via HDBSCAN+UMAP, with selective, drift-driven splitting, merging, and pruning to preserve cluster continuity. Key contributions include LLM-based concern extraction and segmentation, contrastive filtering, metric-driven drift management, and a scalable deployment showing substantial improvements in clustering quality and explainability, including drift narratives and lifecycle role assignment. The approach demonstrates robust performance on a large enterprise corpus, with synthetic data validation supporting generalizability, and offers practical impact for real-time issue tracking and trend detection without full reclustering.

Abstract

Clustering customer chat data is vital for cloud providers handling multi service queries. Traditional methods struggle with overlapping concerns and create broad, static clusters that degrade over time. Reclustering disrupts continuity, making issue tracking difficult. We propose an adaptive system that segments multi turn chats into service specific concerns and incrementally refines clusters as new issues arise. Cluster quality is tracked via DaviesBouldin Index and Silhouette Scores, with LLM based splitting applied only to degraded clusters. Our method improves Silhouette Scores by over 100\% and reduces DBI by 65.6\% compared to baselines, enabling scalable, real time analytics without full reclustering.

LLM-Guided Lifecycle-Aware Clustering of Multi-Turn Customer Support Conversations

TL;DR

This work addresses the challenge of clustering evolving, multi-turn customer support conversations across multiple cloud services. It introduces a lifecycle-aware framework that uses LLMs to segment chats into service-specific concerns, then applies incremental clustering within service groups via HDBSCAN+UMAP, with selective, drift-driven splitting, merging, and pruning to preserve cluster continuity. Key contributions include LLM-based concern extraction and segmentation, contrastive filtering, metric-driven drift management, and a scalable deployment showing substantial improvements in clustering quality and explainability, including drift narratives and lifecycle role assignment. The approach demonstrates robust performance on a large enterprise corpus, with synthetic data validation supporting generalizability, and offers practical impact for real-time issue tracking and trend detection without full reclustering.

Abstract

Clustering customer chat data is vital for cloud providers handling multi service queries. Traditional methods struggle with overlapping concerns and create broad, static clusters that degrade over time. Reclustering disrupts continuity, making issue tracking difficult. We propose an adaptive system that segments multi turn chats into service specific concerns and incrementally refines clusters as new issues arise. Cluster quality is tracked via DaviesBouldin Index and Silhouette Scores, with LLM based splitting applied only to degraded clusters. Our method improves Silhouette Scores by over 100\% and reduces DBI by 65.6\% compared to baselines, enabling scalable, real time analytics without full reclustering.
Paper Structure (51 sections, 2 equations, 17 figures, 22 tables, 1 algorithm)

This paper contains 51 sections, 2 equations, 17 figures, 22 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Our Contributions:
  3. Related Work
  4. Traditional Clustering in Customer Support
  5. Multi-Turn Dialogue Understanding and Intent Classification
  6. Embedding-Based Retrieval
  7. Incremental and Adaptive Clustering
  8. Our Approach
  9. Methodology
  10. Base Cluster Creation
  11. Incremental Clustering
  12. Evolving New Clusters
  13. Cluster Refinement: Splitting Clusters
  14. Drift Narrative Generation
  15. Cluster Lifecycle Management: Merge and Prune
  16. ...and 36 more sections

Figures (17)

  • Figure 1: Architecture: The figure outlines base clustering, incremental clustering and cluster refinement pipelines.
  • Figure 2: Creation of Base Clusters
  • Figure 3: Few-shot classification metrics by Group
  • Figure 4: Incremental Clustering Metrics for "Compute with Cluster Refinement"
  • Figure 5: Incremental Clustering Metrics for "Compute" using no Cluster Refinement
  • ...and 12 more figures