Enhancing the Learning Experience: Using Vision-Language Models to Generate Questions for Educational Videos

TL;DR

This work tackles generating learning-oriented questions from educational videos using vision-language models to boost engagement and knowledge acquisition. It systematically evaluates zero-shot and fine-tuned multimodal models (Video-LLaVA, PG-Video-LLaVA, Video-LLaMA) against a textual baseline (Mistral-7B), using LearningQ and Khan Academy TED-Ed data, a custom Inner-Class-Diff metric, and prompts designed to target different cognitive levels. Key findings show zero-shot models underperform, while fine-tuning improves quality yet leaves challenges in relevance, depth, and modality balance; textual baselines sometimes outperform multimodal approaches, highlighting the need for richer multimodal datasets with time-aligned annotations. The paper highlights gaps in current datasets and calls for larger, more diverse, timestamped educational video datasets and ensemble approaches that combine multimodal content analysis with LLM/VLM capabilities to reduce hallucinations and improve content alignment.

Abstract

Web-based educational videos offer flexible learning opportunities and are becoming increasingly popular. However, improving user engagement and knowledge retention remains a challenge. Automatically generated questions can activate learners and support their knowledge acquisition. Further, they can help teachers and learners assess their understanding. While large language and vision-language models have been employed in various tasks, their application to question generation for educational videos remains underexplored. In this paper, we investigate the capabilities of current vision-language models for generating learning-oriented questions for educational video content. We assess (1) out-of-the-box models' performance; (2) fine-tuning effects on content-specific question generation; (3) the impact of different video modalities on question quality; and (4) in a qualitative study, question relevance, answerability, and difficulty levels of generated questions. Our findings delineate the capabilities of current vision-language models, highlighting the need for fine-tuning and addressing challenges in question diversity and relevance. We identify requirements for future multimodal datasets and outline promising research directions.