Awaking the Slides: A Tuning-free and Knowledge-regulated AI Tutoring System via Language Model Coordination

TL;DR

Slide2Lecture is developed, a tuning-free and knowledge-regulated intelligent tutoring system that can effectively convert an input lecture slide into a structured teaching agenda consisting of a set of heterogeneous teaching actions.

Abstract

The vast pre-existing slides serve as rich and important materials to carry lecture knowledge. However, effectively leveraging lecture slides to serve students is difficult due to the multi-modal nature of slide content and the heterogeneous teaching actions. We study the problem of discovering effective designs that convert a slide into an interactive lecture. We develop Slide2Lecture, a tuning-free and knowledge-regulated intelligent tutoring system that can (1) effectively convert an input lecture slide into a structured teaching agenda consisting of a set of heterogeneous teaching actions; (2) create and manage an interactive lecture that generates responsive interactions catering to student learning demands while regulating the interactions to follow teaching actions. Slide2Lecture contains a complete pipeline for learners to obtain an interactive classroom experience to learn the slide. For teachers and developers, Slide2Lecture enables customization to cater to personalized demands. The evaluation rated by annotators and students shows that Slide2Lecture is effective in outperforming the remaining implementation. Slide2Lecture's online deployment has made more than 200K interaction with students in the 3K lecture sessions. We open source Slide2Lecture's implementation in https://anonymous.4open.science/r/slide2lecture-4210/.

Figures (10)

• Figure 1: Framework overview for Slide2Lecture. A slide file is processed to extract its contents and structure. The Plan subsystem then takes the extracted materials to generate a set of teaching actions. Finally, the Teach module calls the matching scene controller to control the agents, creating an interactive lecturing to serve user learning while aligning with slide.
• Figure 2: Example of slide file segmentation ($f_{seg}$ and $prune$). In each iteration, the LLM is asked to insert a page into the existing agenda. We then prune the nodes that are not direct siblings of the ancestors to limit the context length.
• Figure 3: Evaluation result of $AskQuestion$ generation.
• Figure 4: Results for in-class user study. The users are asked to rate the system after taking a lecture. We present the results in box-plot to display the median (red lines), 25% to 75% range ($IQR$ boxes), and outliers (dots) filtered with whiskers($1.5\times IQR$).
• Figure 5: Precision of scene controller during user study.