Making an agent's trust stable in a series of success and failure tasks through empathy
Takahiro Tsumura, Seiji Yamada
TL;DR
This work investigates how to stabilize human trust in AI agents by manipulating agent empathy and monitoring trust across a five-phase success/failure image-recognition task. Using a two-factor mixed design, with empathy as a between-subjects factor and phase as a within-subjects factor, the study finds a significant interaction indicating empathy helps maintain stable trust over time. However, the expected pattern that trust increases after success and decreases after failure is not consistently supported, suggesting trust calibration is governed by empathy rather than immediate outcomes alone. The results inform design principles for empathetic AI agents to foster appropriate, durable trust in human–agent collaborations across tasks.
Abstract
As AI technology develops, trust in AI agents is becoming more important for more AI applications in human society. Possible ways to improve the trust relationship include empathy, success-failure series, and capability (performance). Appropriate trust is less likely to cause deviations between actual and ideal performance. In this study, we focus on the agent's empathy and success-failure series to increase trust in AI agents. We experimentally examine the effect of empathy from agent to person on changes in trust over time. The experiment was conducted with a two-factor mixed design: empathy (available, not available) and success-failure series (phase 1 to phase 5). An analysis of variance (ANOVA) was conducted using data from 198 participants. The results showed an interaction between the empathy factor and the success-failure series factor, with trust in the agent stabilizing when empathy was present. This result supports our hypothesis. This study shows that designing AI agents to be empathetic is an important factor for trust and helps humans build appropriate trust relationships with AI agents.