Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI-Salesman: Towards Reliable Large Language Model Driven Telemarketing

Qingyu Zhang, Chunlei Xin, Xuanang Chen, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun, Qing Ye, Qianlong Xie, Xingxing Wang

TL;DR

The paper tackles goal-driven persuasive telemarketing with large language models by introducing AI-Salesman, which combines a Bayesian-supervised reinforcement learning objective (via GRPO) with a Dynamic Outline-Guided Agent (DOGA) for turn-level strategy. It additionally releases TeleSalesCorpus, the first real-world-grounded telemarketing dialogue dataset, and proposes a comprehensive offline evaluation framework that uses an LLM (GPT-4) as a judge across six sales capabilities and seven qualitative metrics. Empirical results show that direct reinforcement learning without imitation (GRPO w/o SFT) achieves stronger overall performance than SFT-based baselines, while DOGA substantially improves performance on complex, strategic turns. The work demonstrates practical impact by outperforming baselines in automatic metrics and human evaluations, confirming the value of reasoning-aware training and dynamic, library-driven guidance for reliable, customized persuasive dialogue in real-world telemarketing.

Abstract

Goal-driven persuasive dialogue, exemplified by applications like telemarketing, requires sophisticated multi-turn planning and strict factual faithfulness, which remains a significant challenge for even state-of-the-art Large Language Models (LLMs). A lack of task-specific data often limits previous works, and direct LLM application suffers from strategic brittleness and factual hallucination. In this paper, we first construct and release TeleSalesCorpus, the first real-world-grounded dialogue dataset for this domain. We then propose AI-Salesman, a novel framework featuring a dual-stage architecture. For the training stage, we design a Bayesian-supervised reinforcement learning algorithm that learns robust sales strategies from noisy dialogues. For the inference stage, we introduce the Dynamic Outline-Guided Agent (DOGA), which leverages a pre-built script library to provide dynamic, turn-by-turn strategic guidance. Moreover, we design a comprehensive evaluation framework that combines fine-grained metrics for key sales skills with the LLM-as-a-Judge paradigm. Experimental results demonstrate that our proposed AI-Salesman significantly outperforms baseline models in both automatic metrics and comprehensive human evaluations, showcasing its effectiveness in complex persuasive scenarios.

AI-Salesman: Towards Reliable Large Language Model Driven Telemarketing

TL;DR

The paper tackles goal-driven persuasive telemarketing with large language models by introducing AI-Salesman, which combines a Bayesian-supervised reinforcement learning objective (via GRPO) with a Dynamic Outline-Guided Agent (DOGA) for turn-level strategy. It additionally releases TeleSalesCorpus, the first real-world-grounded telemarketing dialogue dataset, and proposes a comprehensive offline evaluation framework that uses an LLM (GPT-4) as a judge across six sales capabilities and seven qualitative metrics. Empirical results show that direct reinforcement learning without imitation (GRPO w/o SFT) achieves stronger overall performance than SFT-based baselines, while DOGA substantially improves performance on complex, strategic turns. The work demonstrates practical impact by outperforming baselines in automatic metrics and human evaluations, confirming the value of reasoning-aware training and dynamic, library-driven guidance for reliable, customized persuasive dialogue in real-world telemarketing.

Abstract

Goal-driven persuasive dialogue, exemplified by applications like telemarketing, requires sophisticated multi-turn planning and strict factual faithfulness, which remains a significant challenge for even state-of-the-art Large Language Models (LLMs). A lack of task-specific data often limits previous works, and direct LLM application suffers from strategic brittleness and factual hallucination. In this paper, we first construct and release TeleSalesCorpus, the first real-world-grounded dialogue dataset for this domain. We then propose AI-Salesman, a novel framework featuring a dual-stage architecture. For the training stage, we design a Bayesian-supervised reinforcement learning algorithm that learns robust sales strategies from noisy dialogues. For the inference stage, we introduce the Dynamic Outline-Guided Agent (DOGA), which leverages a pre-built script library to provide dynamic, turn-by-turn strategic guidance. Moreover, we design a comprehensive evaluation framework that combines fine-grained metrics for key sales skills with the LLM-as-a-Judge paradigm. Experimental results demonstrate that our proposed AI-Salesman significantly outperforms baseline models in both automatic metrics and comprehensive human evaluations, showcasing its effectiveness in complex persuasive scenarios.

Paper Structure

This paper contains 49 sections, 15 equations, 9 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Telemarketing Scenarios
  3. Task Definition
  4. Evaluation Framework
  5. End-to-End Intelligent Sales System
  6. Data Construction
  7. Stage-1: GRPO Training
  8. Bayesian-Supervised Reasoning ($R_{\text{bayes}}$)
  9. Format Adherence ($R_{\text{format}}$)
  10. Relative Length Consistency ($R_{\text{len}}$)
  11. Semantic Similarity ($R_{\text{sem}}$)
  12. Stage-2: Inference With Dynamic Prompt
  13. Offline Stage: Structured Script Library Construction
  14. Online Stage: Real-time Dialogue Management
  15. Experiments
  16. ...and 34 more sections

Figures (9)

  • Figure 1: Overview of Training and Inference for the AI Salesman.
  • Figure 2: Data Construction Framework Overview.
  • Figure 3: AI Salesman Framework Overview.
  • Figure 4: Key experimental results. (a) Bayesian reward ($R_{\text{bayes}}$) stably raises the upper bound of the semantic similarity reward. (b) DOGA shows decisive advantages in complex, strategic capabilities. (c) Our method's performance scales effectively, with the 32B model offering an optimal trade-off.
  • Figure 5: Knowledge Base Entry Example
  • ...and 4 more figures