Chrysalis: A Unified System for Comparing Active Teaching and Passive Learning with AI Agents in Education

TL;DR

Chrysalis investigates two AI-driven pedagogies—tutoring by an LLM and learning-by-teaching to an LLM—using a within-subject study with 36 university students across two CS courses. The authors implement Chrysalis with GPT-4o to support both modes and perform a direct comparison of user experience, intellectual humility, engagement, and learning outcomes. They find no significant overall preference but observe higher intellectual humility and more elaborate discourse in tutoring, alongside longer, more engaged conversations in learning-by-teaching. This work demonstrates the feasibility of a unified platform for cross-mode education research and highlights design considerations for student-owned learning through AI agents.

Abstract

AI-assisted learning has seen a remarkable uptick over the last few years, mainly due to the rise in popularity of Large Language Models (LLMs). Their ability to hold long-form, natural language interactions with users makes them excellent resources for exploring school- and university-level topics in a dynamic, active manner. We compare students' experiences when interacting with an LLM companion in two capacities: tutored learning and learning-by-teaching. We do this using Chrysalis, an LLM-based system that we have designed to support both AI tutors and AI teachable agents for any topic. Through a within-subject exploratory study with 36 participants, we present insights into student preferences between the two strategies and how constructs such as intellectual humility vary between these two interaction modes. To our knowledge, we are the first to conduct a direct comparison study on the effects of using an LLM as a tutor versus as a teachable agent on multiple topics. We hope that our work opens up new avenues for future research in this area.

Figures (7)

• Figure 1: The interface that our participants were exposed to. This is a modification of Chrysalis that is designed to include the other components of our study. Here, the user is teaching the LLM student about Transformers. The lesson plan can be seen on the lower left-hand side.
• Figure 2: Our counterbalanced design. For each class, there are two groups (1 and 2) which participants are added to in an alternating manner. Both groups are exposed to the same topics, but the topic which the student teaches the agent (learning-by-teaching, abbreviated to LbT in the figure), and the topic which they are taught by the agent (AI Tutor), depends on their group.
• Figure 3: Differences in proportion of intellectual humility across all messages between AI tutoring and learning-by-teaching.
• Figure 4: Part of speech distribution across message in learning-by-teaching (left) and tutoring (right).
• Figure 5: Difference in part-of-speech percentage between learning-by-teaching and tutoring.