On-Demand Instructional Material Providing Agent Based on MLLM for Tutoring Support

Abstract

Effective instruction in tutoring requires promptly providing instructional materials that match the needs of each student (e.g., in response to questions). In this study, we introduce an agent that automatically delivers supplementary materials on demand during one-on-one tutoring sessions. Our agent uses a multimodal large language model to analyze spoken dialogue between the instructor and the student, automatically generate search queries, and retrieve relevant Web images. Evaluation experiments demonstrate that our agent reduces the average image retrieval time by 44.4 s compared to cases without support and successfully provides images of acceptable quality in 85.7% of trials. These results indicate that our agent effectively supports instructors during tutoring sessions.

Figures (3)

• Figure 1: Proposed Method
• Figure 2: Overview of the System Operation Process
• Figure 3: Prompt Used to Retrieve Image Search Query