AI instructional agent improves student's perceived learner control and learning outcome: empirical evidence from a randomized controlled trial

TL;DR

This study tackles the challenge of enhancing learner agency and achievement in scalable education by comparing an AI instructional agent, a human instructor, and a self-paced MOOC with chatbot support in a randomized controlled trial. The MAIC-based AI agent delivers lectures while allowing real-time questioning, enabling users to control pacing and interaction. Results show that AI training increases perceived learner control and post-test performance, with PLC positively predicting learning outcomes; behavioral data reveal more frequent interactions and more efficient time use in the AI condition. The findings suggest that integrating structured content delivery with real-time, user-initiated interaction can sustain autonomy without overwhelming learners, offering a scalable design for personalized instruction in higher education.

Abstract

This study examines the impact of an AI instructional agent on students' perceived learner control and academic performance in a medium demanding course with lecturing as the main teaching strategy. Based on a randomized controlled trial, three instructional conditions were compared: a traditional human teacher, a self-paced MOOC with chatbot support, and an AI instructional agent capable of delivering lectures and responding to questions in real time. Students in the AI instructional agent group reported significantly higher levels of perceived learner control compared to the other groups. They also completed the learning task more efficiently and engaged in more frequent interactions with the instructional system. Regression analyzes showed that perceived learner control positively predicted post-test performance, with behavioral indicators such as reduced learning time and higher interaction frequency supporting this relationship. These findings suggest that AI instructional agents, when designed to support personalized pace and responsive interaction, can enhance both students' learning experience and learning outcomes.

Figures (3)

• Figure 1: RCT Design Diagram
• Figure 2: The Comparison between Three Groups of Key Variables
• Figure 3: The Correlation between Posttest and Other Variables