Debt Collection Negotiations with Large Language Models: An Evaluation System and Optimizing Decision Making with Multi-Agent

TL;DR

This work addresses automating debt collection negotiations (DCN) with large language models (LLMs) by constructing a synthetic 975-record DCN dataset and a 13-metric evaluation framework to assess both dialogue quality and financial outcomes. It reveals that vanilla LLMs tend to concede excessively and struggle with decision rationality, prompting the design of the Multi-Agent Debt Negotiation (MADeN) framework, which adds Planning and Judging modules to improve strategy and evaluation. The authors also explore post-training approaches, including Direct Preference Optimization with rejection sampling, demonstrating that MADeN and DPO-MAG can substantially improve debt recovery, collection efficiency, and debtor health relative to baseline LLMs. Together, these contributions advance AI-assisted DCN and provide a benchmark for future research on autonomous negotiation in finance.

Abstract

Debt collection negotiations (DCN) are vital for managing non-performing loans (NPLs) and reducing creditor losses. Traditional methods are labor-intensive, while large language models (LLMs) offer promising automation potential. However, prior systems lacked dynamic negotiation and real-time decision-making capabilities. This paper explores LLMs in automating DCN and proposes a novel evaluation framework with 13 metrics across 4 aspects. Our experiments reveal that LLMs tend to over-concede compared to human negotiators. To address this, we propose the Multi-Agent Debt Negotiation (MADeN) framework, incorporating planning and judging modules to improve decision rationality. We also apply post-training techniques, including DPO with rejection sampling, to optimize performance. Our studies provide valuable insights for practitioners and researchers seeking to enhance efficiency and outcomes in this domain.

Figures (10)

• Figure 1: An Example of a Debt Collection Negotiation (DCN). On the left and right sides are the information cards representing the data controlled by the debtor and the creditor, respectively. The black text represents the basic debt information, while the red text represents the debtor’s personal financial information. In the center, we demonstrate the use of LLM-based agents to simulate the dialogue. Each time, both the debtor and the creditor output a set of (Thoughts, Dialogue, Action). Thoughts refers to their internal thought process, visible only to themselves; Dialogue represents the conversation in natural language; and Action refers to the specific activities represented in a formal language within the dialogue. Each negotiation consists of multiple rounds of such interactions, ultimately leading to the negotiation outcome. The English text was automatically translated using Google Translate.
• Figure 2: The future trajectories of the debtor’s remaining assets and outstanding debt under three installment plans (6, 12, and 18 months from left to right) are shown, with all other variables held constant. The 6-month plan causes the debtor’s assets to fall below zero, making repayment impossible. In contrast, the 12-month and 18-month plans maintain a healthy asset level, though the 18-month plan significantly reduces recovery efficiency. The 12-month plan is the most balanced solution. Different background colors represent five difficulty tiers, with Tier 1 being the most challenging. The specific ranges and descriptions of the tiers are provided in Appendix \ref{['app:diff_cat']}.
• Figure 3: Evaluation system of DCN.
• Figure 4: MADeN Framework overview.
• Figure 5: Reject Sampling Process. D.Agent and C.Agent represent the debtor agent and creditor agent. Creditor agent can be designed in two forms, depending on the use of the MADeN framework (DG and MAG).