Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond IVR Touch-Tones: Customer Intent Routing using LLMs

Sergio Rojas-Galeano

TL;DR

The paper addresses the inefficiency of touch-tone IVRs by exploring natural language routing with Large Language Models. It introduces a synthetic, three-model pipeline that generates a 23-node IVR, 920 user intents (230 base + 690 augmented), and routing evaluations under descriptive and flattened context representations. Results show that Flattened IVR Paths achieve higher accuracy (up to 89.13% on the Base dataset) than Descriptive Menu representations, while linguistic augmentation adds noise and can reduce precision. The work demonstrates a proof-of-concept for LLM-based IVR routing and provides design guidance for building more natural, scalable phone-based customer support systems.

Abstract

Widespread frustration with rigid touch-tone Interactive Voice Response (IVR) systems for customer service underscores the need for more direct and intuitive language interaction. While speech technologies are necessary, the key challenge lies in routing intents from user phrasings to IVR menu paths, a task where Large Language Models (LLMs) show strong potential. Progress, however, is limited by data scarcity, as real IVR structures and interactions are often proprietary. We present a novel LLM-based methodology to address this gap. Using three distinct models, we synthesized a realistic 23-node IVR structure, generated 920 user intents (230 base and 690 augmented), and performed the routing task. We evaluate two prompt designs: descriptive hierarchical menus and flattened path representations, across both base and augmented datasets. Results show that flattened paths consistently yield higher accuracy, reaching 89.13% on the base dataset compared to 81.30% with the descriptive format, while augmentation introduces linguistic noise that slightly reduces performance. Confusion matrix analysis further suggests that low-performing routes may reflect not only model limitations but also redundancies in menu design. Overall, our findings demonstrate proof-of-concept that LLMs can enable IVR routing through a smoother, more seamless user experience -- moving customer service one step ahead of touch-tone menus.

Beyond IVR Touch-Tones: Customer Intent Routing using LLMs

TL;DR

The paper addresses the inefficiency of touch-tone IVRs by exploring natural language routing with Large Language Models. It introduces a synthetic, three-model pipeline that generates a 23-node IVR, 920 user intents (230 base + 690 augmented), and routing evaluations under descriptive and flattened context representations. Results show that Flattened IVR Paths achieve higher accuracy (up to 89.13% on the Base dataset) than Descriptive Menu representations, while linguistic augmentation adds noise and can reduce precision. The work demonstrates a proof-of-concept for LLM-based IVR routing and provides design guidance for building more natural, scalable phone-based customer support systems.

Abstract

Widespread frustration with rigid touch-tone Interactive Voice Response (IVR) systems for customer service underscores the need for more direct and intuitive language interaction. While speech technologies are necessary, the key challenge lies in routing intents from user phrasings to IVR menu paths, a task where Large Language Models (LLMs) show strong potential. Progress, however, is limited by data scarcity, as real IVR structures and interactions are often proprietary. We present a novel LLM-based methodology to address this gap. Using three distinct models, we synthesized a realistic 23-node IVR structure, generated 920 user intents (230 base and 690 augmented), and performed the routing task. We evaluate two prompt designs: descriptive hierarchical menus and flattened path representations, across both base and augmented datasets. Results show that flattened paths consistently yield higher accuracy, reaching 89.13% on the base dataset compared to 81.30% with the descriptive format, while augmentation introduces linguistic noise that slightly reduces performance. Confusion matrix analysis further suggests that low-performing routes may reflect not only model limitations but also redundancies in menu design. Overall, our findings demonstrate proof-of-concept that LLMs can enable IVR routing through a smoother, more seamless user experience -- moving customer service one step ahead of touch-tone menus.
Paper Structure (16 sections, 4 figures, 2 tables)

This paper contains 16 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Experimental Design Overview
  5. Synthetic IVR Menu Generation
  6. User Intent Dataset Creation
  7. Natural Language Intent Routing
  8. Mitigation of Data Contamination Bias
  9. Evaluation Metrics
  10. Results
  11. Overall Routing Accuracy
  12. Confusion Matrix Insights
  13. Discussion
  14. Key Insights
  15. Limitations and Future Work
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Confusion matrix for Descriptive IVR Menu with Augmented Dataset.
  • Figure 2: Confusion matrix for Descriptive IVR Menu with Base Only Dataset.
  • Figure 3: Confusion matrix for Flattened IVR Paths with Augmented Dataset.
  • Figure 4: Confusion matrix for Flattened IVR Paths with Base Only Dataset.