TickIt: Leveraging Large Language Models for Automated Ticket Escalation

TL;DR

TickIt addresses the challenge of scalable, accurate ticket escalations in large cloud platforms by deploying an online, topic-aware escalation framework powered by large language models. It jointly handles multi-class ticket classification, cross-ticket deduplication via embedding similarity, and category-guided fine-tuning driven by analyst feedback, with CoT/ICL prompting and LoRA-based SFT. In a real production setting on Volcano Engine, TickIt processed over 20k tickets and achieved significant improvements in escalation accuracy and MTTR reduction (about 39%), while reducing redundant escalations through a deduplication mechanism that leverages issue rewriting. The work demonstrates that online, feedback-driven LLM fine-tuning and thoughtful data augmentation can yield robust, scalable ticket escalation for complex, high-volume cloud environments.

Abstract

In large-scale cloud service systems, support tickets serve as a critical mechanism for resolving customer issues and maintaining service quality. However, traditional manual ticket escalation processes encounter significant challenges, including inefficiency, inaccuracy, and difficulty in handling the high volume and complexity of tickets. While previous research has proposed various machine learning models for ticket classification, these approaches often overlook the practical demands of real-world escalations, such as dynamic ticket updates, topic-specific routing, and the analysis of ticket relationships. To bridge this gap, this paper introduces TickIt, an innovative online ticket escalation framework powered by Large Language Models. TickIt enables topic-aware, dynamic, and relationship-driven ticket escalations by continuously updating ticket states, assigning tickets to the most appropriate support teams, exploring ticket correlations, and leveraging category-guided supervised fine-tuning to continuously improve its performance. By deploying TickIt in ByteDance's cloud service platform Volcano Engine, we validate its efficacy and practicality, marking a significant advancement in the field of automated ticket escalation for large-scale cloud service systems.

Figures (6)

• Figure 1: Overview framework of TickIt.
• Figure 2: Classification prompt for customer tickets escalation
• Figure 3: Customer ticket state within its lifecycle
• Figure 4: Prompt for escalation deduplication.
• Figure 5: Data augmentation for labeled data.