Who You Explain To Matters: Learning by Explaining to Conversational Agents with Different Pedagogical Roles

TL;DR

This study investigates how four pedagogical roles for conversational agents—Tutee, Peer, Challenger, and a Control condition—influence learning-by-explaining in an economics task. Using a between-subjects design with 96 participants, the authors examine moment-to-moment interaction patterns, learning outcomes, and experiential measures during five rounds of explanation about supply and demand. Results show that agent roles shape explanatory behavior and affective experience (e.g., higher cognitive investment with Tutee, greater absorption with Peer and Challenger, and elevated pressure with Tutee), but do not yield significant differences in objective post-test scores. The findings argue for role-specific agent design to support distinct pedagogical goals and learning phases, highlighting the importance of metacognitive scaffolding and balanced human–agent co-responsibility for sustained engagement and deeper reasoning over time.

Abstract

Conversational agents are increasingly used in education for learning support. An application is "learning by explaining", where learners explain their understanding to an agent. However, existing research focuses on single roles, leaving it unclear how different pedagogical roles influence learners' interaction patterns, learning outcomes and experiences. We conducted a between-subjects study (N=96) comparing agents with three pedagogical roles (Tutee, Peer, Challenger) and a control condition while learning an economics concept. We found that different pedagogical roles shaped learning dynamics, including interaction patterns and experiences. Specifically, the Tutee agent elicited the most cognitive investment but led to high pressure. The Peer agent fostered high absorption and interest through collaborative dialogue. The Challenger agent promoted cognitive and metacognitive acts, enhancing critical thinking with moderate pressure. The findings highlight how agent roles shape different learning dynamics, guiding the design of educational agents tailored to specific pedagogical goals and learning phases.

Figures (6)

• Figure 1: Workflows of the three agent roles (Tutee, Peer, and Challenger). The figure illustrates the decision flow and key actions for each role. Note that the agent responses shown in the figure are only schematic examples of the response structure; in our study, the agent responses under each condition were constrained to around 80 words.
• Figure 2: Interaction interface used in the study under the Tutee condition. The left panel displays the task problem, while the right panel shows the chat interface where participants explained to the agent.
• Figure 3: Experimental procedure. Participants read instructions and gave consent, completed a pre-test (5 min), studied learning materials (15 min), engaged in five rounds of interaction with agents (20 min), and finished with a post-test and surveys (20 min).
• Figure 4: User interaction metrics across agent roles. (a) Word counts per conversation; (b) Frequency of reviewing material; (c) Duration of reviewing material. Means ($M$) and standard deviations ($SD$) are displayed on the x-axis for each condition. The error bars represent the standard error. Significance brackets indicate pairwise post-hoc differences; $^{*}p<.05$, $^{**}p<.01$, $^{***}p<.001$.
• Figure 5: Turn-by-turn distribution of coded user acts across the five interaction rounds for each role (Tutee, Peer, Challenger, Control). For readability, each subplot only displays act types that appeared at least 3 times within the turn (>10% of user messages).