Do Expressions Change Decisions? Exploring the Impact of AI's Explanation Tone on Decision-Making

TL;DR

This work investigates how the tone of AI explanations influences human decision-making across three AI roles—assistant, second-opinion provider, and expert—using three online experiments with tone manipulations (neutral vs. intervention tones). Across scenarios, tone effects emerged most strongly in the opinion formation task, where stance and confidence shifted with tone, while recommendation showed mixed effects and advice revealed nuanced confidence changes and gender interactions. The study demonstrates that user attributes, especially age, modulate tone influence, whereas extroversion shows limited impact; qualitative data highlight expectations for clarity and empathy but also concerns about manipulation and inconsistency. Collectively, the findings offer design guidance for tone-enabled explanations, balancing user experience with ethical safeguards and avoiding biases across user groups and AI roles.

Abstract

Explanatory information helps users to evaluate the suggestions offered by AI-driven decision support systems. With large language models, adjusting explanation expressions has become much easier. However, how these expressions influence human decision-making remains largely unexplored. This study investigated the effect of explanation tone (e.g., formal or humorous) on decision-making, focusing on AI roles and user attributes. We conducted user experiments across three scenarios depending on AI roles (assistant, second-opinion provider, and expert) using datasets designed with varying tones. The results revealed that tone significantly influenced decision-making regardless of user attributes in the second-opinion scenario, whereas its impact varied by user attributes in the assistant and expert scenarios. In addition, older users were more influenced by tone, and highly extroverted users exhibited discrepancies between their perceptions and decisions. Furthermore, open-ended questionnaires highlighted that users expect tone adjustments to enhance their experience while emphasizing the importance of tone consistency and ethical considerations. Our findings provide crucial insights into the design of explanation expressions.

Figures (18)

• Figure 1: User Study Procedure. We conducted user studies consisting of six steps. Step 1: participants complete a pre-questionnaire to assess their demographics (age and gender) and Big Five personality traits. Step 2: participants are asked to perform tasks without any presentation of explanations, referred to as the initial phase. Step 3: participants complete the tasks with explanations in a neutral tone, referred to as the pre-intervention phase. Step 4: participants answer an open-ended questions to assess how much the explanations affected their tasks. Step 5: participants perform the tasks with explanations in an assigned intervention tone, referred to as the intervention phase. Step 6: participants are asked to answer open-ended questions to assess how much intervening explanations affect the tasks. The tasks presented to participants were identical across all three phases, ensuring comparability of responses.
• Figure 2: Recommendation scenario: Screen presented to users in each phase. AI-generated advertisements were provided in a neutral tone in Phase 2 and in an intervention tone in Phase 3.
• Figure 3: Recommendation scenario: Distributions of score differences by tone (Phases 2 to 3). The violin and box plots show $d_u^{\rm tone}$ across five tone interventions.
• Figure 4: Recommendation scenario: Interaction of tone and gender on score differences (Phases 2 to 3). The plot illustrates the distribution of $d_u^{\rm tone}$ across tone interventions, divided by gender (male and female).
• Figure 5: Recommendation scenario: Correlation between age, personality traits, and score differences by tone (Phase 2 to 3). The top-left graph shows the relationship between age and $d_u^{\rm tone}$, whereas the remaining graphs show the relationship for personality traits.