Thinking Assistants: LLM-Based Conversational Assistants that Help Users Think By Asking rather than Answering

TL;DR

The paper introduces Thinking Assistants, a new class of LLM-based conversational agents designed to promote productive self-reflection by combining reflective inquiry with domain-specific expert knowledge. Through a lab experiment (N=80) and a longitudinal deployment (Gradschool.chat) with 223 conversations, the authors show that informed inquiry—questions grounded in expertise—elicits more reasoning than baseline modes that only ask or provide advice. The work presents a concrete implementation (Gradschool.chat) that integrates reflective prompting, expert knowledge from professors, and a safety verifier to curb inaccuracies, followed by in-the-wild evaluation and qualitative analysis. Findings highlight the potential of this modality to improve academic advising and high-stakes decision-making while underscoring challenges in user mental models, safety, and scalability, outlining design implications and future directions for robust, reflective AI assistants.

Abstract

Many AI systems focus solely on providing solutions or explaining outcomes. However, complex tasks like research and strategic thinking often benefit from a more comprehensive approach to augmenting the thinking process rather than passively getting information. We introduce the concept of "Thinking Assistant", a new genre of assistants that help users improve decision-making with a combination of asking reflection questions based on expert knowledge. Through our lab study (N=80), these Large Language Model (LLM) based Thinking Assistants were better able to guide users to make important decisions, compared with conversational agents that only asked questions, provided advice, or neither. Based on the results, we develop a Thinking Assistant in academic career development, determining research trajectory or developing one's unique research identity, which requires deliberation, reflection and experts' advice accordingly. In a longitudinal deployment with 223 conversations, participants responded positively to approximately 65% of the responses. Our work proposes directions for developing more effective LLM agents. Rather than adhering to the prevailing authoritative approach of generating definitive answers, LLM agents aimed at assisting with cognitive enhancement should prioritize fostering reflection. They should initially provide responses designed to prompt thoughtful consideration through inquiring, followed by offering advice only after gaining a deeper understanding of the user's context and needs.

Figures (4)

• Figure 1: Interfaces used for the lab study (Left): For the "diary" group, below the discussion prompt and instructions, there is an empty textbox where participants can write. (Right): For the other group, under the discussion prompt, there are instructions for the chatbot task and chat interface with a chatbot.
• Figure 2: Gradschool.chat is a Thinking Assistant that helps junior researchers reflect on research interests. Based on the user's question, the assistant chooses between a 'probing' mode, understanding user interests and prompting reflection, and an 'answering' mode that offers relevant information. Most conversational turns end with an encouraging follow-up question.
• Figure 3: Gradschool.chat interface. Left: Before a user initiates a conversation, it shows the description of the chatbot. In the interface, we suggest users start by sharing their research interests. Right: The chatbot primarily focuses on asking thought-provoking questions and probing users' research interests. Every three turns of messages, the interface asks users for binary feedback before they can continue the conversation.
• Figure 4: Safety bot: This bot is solely designed to correct critical factual information regarding the professor. We found that such a secondary bot improved factual safety, while suggesting an understandable mental model to users.