Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Diverse and Effective Retrieval-Based Debt Collection System with Expert Knowledge

Jiaming Luo, Weiyi Luo, Guoqing Sun, Mengchen Zhu, Haifeng Tang, Kunyao Lan, Mengyue Wu, Kenny Q. Zhu

TL;DR

Debt collection is costly and traditional rule-based systems struggle with script diversity and adaptability. The authors propose SCORES, a two-stage retrieval framework that first auto-generates a diverse script library from real conversations and then selects responses via a recall model followed by an LLM-based ranking, with knowledge distillation to smaller models for efficiency. Key contributions include automatic production of 1,000+ scripts across 9 strategies, a substantial recall and ranking performance boost, and a scalable deployment framework adaptable to related domains. The approach yields practical gains on bank data, demonstrating how retrieval-based chatbots can enhance efficiency and effectiveness in finance and customer service.

Abstract

Designing effective debt collection systems is crucial for improving operational efficiency and reducing costs in the financial industry. However, the challenges of maintaining script diversity, contextual relevance, and coherence make this task particularly difficult. This paper presents a debt collection system based on real debtor-collector data from a major commercial bank. We construct a script library from real-world debt collection conversations, and propose a two-stage retrieval based response system for contextual relevance. Experimental results show that our system improves script diversity, enhances response relevance, and achieves practical deployment efficiency through knowledge distillation. This work offers a scalable and automated solution, providing valuable insights for advancing debt collection practices in real-world applications.

A Diverse and Effective Retrieval-Based Debt Collection System with Expert Knowledge

TL;DR

Debt collection is costly and traditional rule-based systems struggle with script diversity and adaptability. The authors propose SCORES, a two-stage retrieval framework that first auto-generates a diverse script library from real conversations and then selects responses via a recall model followed by an LLM-based ranking, with knowledge distillation to smaller models for efficiency. Key contributions include automatic production of 1,000+ scripts across 9 strategies, a substantial recall and ranking performance boost, and a scalable deployment framework adaptable to related domains. The approach yields practical gains on bank data, demonstrating how retrieval-based chatbots can enhance efficiency and effectiveness in finance and customer service.

Abstract

Designing effective debt collection systems is crucial for improving operational efficiency and reducing costs in the financial industry. However, the challenges of maintaining script diversity, contextual relevance, and coherence make this task particularly difficult. This paper presents a debt collection system based on real debtor-collector data from a major commercial bank. We construct a script library from real-world debt collection conversations, and propose a two-stage retrieval based response system for contextual relevance. Experimental results show that our system improves script diversity, enhances response relevance, and achieves practical deployment efficiency through knowledge distillation. This work offers a scalable and automated solution, providing valuable insights for advancing debt collection practices in real-world applications.

Paper Structure

This paper contains 25 sections, 6 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Automatic Script Library Construction
  4. Data Preparation
  5. Seed Scripts Selection
  6. Script Generation
  7. Retrieval-based Response System
  8. Recall Stage
  9. Ranking Stage
  10. Experiments
  11. Datasets
  12. Implementation Details
  13. Script Library Construction
  14. Response System Construction
  15. Results and Discussions
  16. ...and 10 more sections

Figures (3)

  • Figure 1: An exemplar between a debtor and a collector, with three candidate responses. The debtor's intent is labeled in red while the strategies in collector's responses are labeled in blue.
  • Figure 2: Overview of the SCORES framework. A script library is constructed from chat logs, followed by a two-stage response selection system.
  • Figure 3: The prompt used for LLM evaluation.