Anger Speaks Louder? Exploring the Effects of AI Nonverbal Emotional Cues on Human Decision Certainty in Moral Dilemmas

TL;DR

This work investigates whether nonverbal emotional cues from AI, delivered via AniBalloons, can modulate human certainty in moral dilemmas. Using a two‑round online study with anger, sadness, and a verbal baseline, the authors show that anger cues can trigger reversal shifts in decision certainty, with gender shaping perceived AI influence. The findings suggest that subtle nonverbal cues can meaningfully affect moral decision processes and user perceptions, offering design opportunities while raising ethical concerns about manipulation. Overall, the study demonstrates a practical method to augment AI influence in decision contexts and highlights the need for careful consideration of gender dynamics in human–AI interactions.

Abstract

Exploring moral dilemmas allows individuals to navigate moral complexity, where a reversal in decision certainty, shifting toward the opposite of one's initial choice, could reflect open-mindedness and less rigidity. This study probes how nonverbal emotional cues from conversational agents could influence decision certainty in moral dilemmas. While existing research heavily focused on verbal aspects of human-agent interaction, we investigated the impact of agents expressing anger and sadness towards the moral situations through animated chat balloons. We compared these with a baseline where agents offered same responses without nonverbal cues. Results show that agents displaying anger significantly caused reversal shifts in decision certainty. The interaction between participant gender and agents' nonverbal emotional cues significantly affects participants' perception of AI's influence. These findings reveal that even subtly altering agents' nonverbal cues may impact human moral decisions, presenting both opportunities to leverage these effects for positive outcomes and ethical risks for future human-AI systems.

Figures (5)

• Figure 1: Task Process: Each user is asked to read the scenario and make decisions in two rounds with an AI assistant. In the second round, the AI assistant provides suggestions about the dilemma wrapped in an emotional bubble.
• Figure 2: AniBalloons used in this experiment. The left-side column features two animations representing anger, while the right-side column features two animations representing sadness.
• Figure 3: User interface example for decision-making with the AI assistant with an emotional anger cue expresses anger. I marks the dilemma scenario, II marks the AI assistant and its verbal response, and III marks the decision-making area, including options for leaving comments, selecting a decision, and rating certainty.
• Figure 4: Impact of nonverbal emotional cues on Reversal Certainty Shift(RCS). Significant differences are indicated between groups(*: p < 0.05, **: p < 0.01). Error bars represent 95% confidence intervals.
• Figure 5: Impact of nonverbal emotional cues on the AI assistant's influence on participants' moral thinking. Significant differences are indicated between groups(*: p < 0.05, **: p < 0.01). Error bars represent 95% confidence intervals.