Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TEA-Bench: A Systematic Benchmarking of Tool-enhanced Emotional Support Dialogue Agent

Xingyu Sui, Yanyan Zhao, Yulin Hu, Jiahe Guo, Weixiang Zhao, Bing Qin

TL;DR

TEA-Bench introduces the first interactive benchmark for tool-augmented emotional support dialogue agents, integrating realistic scenarios, a MCP-style tool environment, and process-level metrics to jointly assess empathy and factual grounding. It also provides TEA-Dialog, a dataset of grounded tool-enhanced ESC dialogues, enabling analysis of tool usage patterns and training effects. Experiments across nine LLMs show that tool augmentation generally improves emotional support quality and reduces hallucinations, with gains that depend on model capacity and tool-invocation proficiency. The work reveals challenges in robust generalization when fine-tuning on high-quality tool-grounded data, highlighting the need for more sophisticated training and alignment approaches for reliable tool-enabled ESC.

Abstract

Emotional Support Conversation requires not only affective expression but also grounded instrumental support to provide trustworthy guidance. However, existing ESC systems and benchmarks largely focus on affective support in text-only settings, overlooking how external tools can enable factual grounding and reduce hallucination in multi-turn emotional support. We introduce TEA-Bench, the first interactive benchmark for evaluating tool-augmented agents in ESC, featuring realistic emotional scenarios, an MCP-style tool environment, and process-level metrics that jointly assess the quality and factual grounding of emotional support. Experiments on nine LLMs show that tool augmentation generally improves emotional support quality and reduces hallucination, but the gains are strongly capacity-dependent: stronger models use tools more selectively and effectively, while weaker models benefit only marginally. We further release TEA-Dialog, a dataset of tool-enhanced ESC dialogues, and find that supervised fine-tuning improves in-distribution support but generalizes poorly. Our results underscore the importance of tool use in building reliable emotional support agents.

TEA-Bench: A Systematic Benchmarking of Tool-enhanced Emotional Support Dialogue Agent

TL;DR

TEA-Bench introduces the first interactive benchmark for tool-augmented emotional support dialogue agents, integrating realistic scenarios, a MCP-style tool environment, and process-level metrics to jointly assess empathy and factual grounding. It also provides TEA-Dialog, a dataset of grounded tool-enhanced ESC dialogues, enabling analysis of tool usage patterns and training effects. Experiments across nine LLMs show that tool augmentation generally improves emotional support quality and reduces hallucinations, with gains that depend on model capacity and tool-invocation proficiency. The work reveals challenges in robust generalization when fine-tuning on high-quality tool-grounded data, highlighting the need for more sophisticated training and alignment approaches for reliable tool-enabled ESC.

Abstract

Emotional Support Conversation requires not only affective expression but also grounded instrumental support to provide trustworthy guidance. However, existing ESC systems and benchmarks largely focus on affective support in text-only settings, overlooking how external tools can enable factual grounding and reduce hallucination in multi-turn emotional support. We introduce TEA-Bench, the first interactive benchmark for evaluating tool-augmented agents in ESC, featuring realistic emotional scenarios, an MCP-style tool environment, and process-level metrics that jointly assess the quality and factual grounding of emotional support. Experiments on nine LLMs show that tool augmentation generally improves emotional support quality and reduces hallucination, but the gains are strongly capacity-dependent: stronger models use tools more selectively and effectively, while weaker models benefit only marginally. We further release TEA-Dialog, a dataset of tool-enhanced ESC dialogues, and find that supervised fine-tuning improves in-distribution support but generalizes poorly. Our results underscore the importance of tool use in building reliable emotional support agents.
Paper Structure (78 sections, 6 equations, 19 figures, 9 tables)

This paper contains 78 sections, 6 equations, 19 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Emotional Support Conversation.
  4. Benchmarks for Tool-augmented LLM.
  5. TEA-Bench
  6. Task Definition
  7. Asymmetric Information Access.
  8. Evaluation Objectives.
  9. Scenario Construction
  10. Scenario Filtering.
  11. Latent Context Generation.
  12. Context Grounding.
  13. Human Validation.
  14. Tool Environment
  15. Tool Design.
  16. ...and 63 more sections

Figures (19)

  • Figure 1: Comparison of affective-only support vs. instrumental support. (a) Affective-only response hallucinates details, undermining trust. (b) Instrumental support response provides verified, actionable suggestions.
  • Figure 2: Example illustrating a tool-augmented ESC under our benchmark setting. The assistant dynamically invokes external tools to gather contextual information, enabling it to offer emotionally resonant and actionable support rather than generic reassurance.
  • Figure 3: Overview of TEA-Scenario construction pipeline. We filter emotionally rich scenarios, generate latent spatiotemporal attributes via LLM, ground them through map-based APIs, and validate through human review.
  • Figure 4: Overview of TEA-Bench. The evaluated agent engages in multi-turn emotional support dialogues with a simulated user and may invoke external tools via the Model Context Protocol (MCP). A Hallucination Detection Module verifies factual grounding in agent responses based on dialogue history and tool observations. Complete dialogues are evaluated using the TEA score and factuality metrics.
  • Figure 5: Average number of tool calls per dialogue across different models on TEA-Bench.
  • ...and 14 more figures