Watching AI Think: User Perceptions of Visible Thinking in Chatbots

TL;DR

The paper investigates how visible thinking in chatbots affects user perceptions in emotionally sensitive, help-seeking contexts. Using a $3×2$ mixed design, it manipulates Thinking Content (None, Emotionally-Supportive, Expertise-Supportive) and Conversation Context (Habit-related vs Feelings-related) while participants interact with a GPT-4o-based chatbot. Quantitative results show that Emotionally-Supportive thinking increases emotional responsiveness and warmth, while Expertise-Supportive thinking enhances understanding and trust, with Feelings contexts amplifying emotional responses; none of the effects interact significantly. Qualitative data reveal nuanced reactions, including expectancy violations when thinking content and final responses are incongruent and varied judgments about authenticity. The findings offer practical guidance for designing visible thinking cues that balance affective support with credibility in sensitive AI-assisted dialogues, and highlight the importance of aligning thinking displays with user needs and context in real-world deployments.

Abstract

People increasingly turn to conversational agents such as ChatGPT to seek guidance for their personal problems. As these systems grow in capability, many now display elements of "thinking": short reflective statements that reveal a model's intentions or values before responding. While initially introduced to promote transparency, such visible thinking can also anthropomorphise the agent and shape user expectations. Yet little is known about how these displays affect user perceptions in help-seeking contexts. We conducted a 3 x 2 mixed design experiment examining the impact of 'Thinking Content' (None, Emotionally-Supportive, Expertise-Supportive) and 'Conversation Context' (Habit-related vs. Feelings-related problems) on users' perceptions of empathy, warmth, competence, and engagement. Participants interacted with a chatbot that either showed no visible thinking or presented value-oriented reflections prior to its response. Our findings contribute to understanding how thinking transparency influences user experience in supportive dialogues, and offer implications for designing conversational agents that communicate intentions in sensitive, help-seeking scenarios.

Figures (5)

• Figure 1: Experiment Flow: Participants were assigned to one of three Thinking Content conditions, and shared both Habit- and Feelings-related problems with the chatbot (order randomised). After each interaction and at the end of the study, participants provided evaluations including survey responses and open-ended feedback.
• Figure 2: Chatbot Interaction: The interaction flow with the chatbot. Example shows someone sharing a Habit-related problem with Emotionally-Supportive condition.
• Figure 3: The instructions used for the two Conversation Context conditions.
• Figure 4: Bar plots showing the three measures of PETS by Thinking Content condition (Emotionally-Supportive, Expertise-Supportive, None). Error bars represent standard errors. Asterisks indicate statistically significant differences between conditions.
• Figure 5: Bar plots showing Warmth, Competence, and Respect by Thinking Content condition (Emotionally-Supportive, Expertise-Supportive, None). Error bars represent standard errors. Asterisks denote statistically significant differences.