Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SlideItRight: Using AI to Find Relevant Slides and Provide Feedback for Open-Ended Questions

Chloe Qianhui Zhao, Jie Cao, Eason Chen, Kenneth R. Koedinger, Jionghao Lin

TL;DR

SlideItRight tackles the challenge of scalable, personalized feedback by combining LLM-generated textual feedback with retrieved, slide-grounded references. The system employs retrieval-augmented generation and a 2×2 crowdsourced study to compare human, AI, slide, and combined feedback modalities, measuring pre/post learning gains and post-survey perceptions. Results show significant learning gains across all conditions but no statistically significant differences between modalities; AI offers strong personalization and rapid feedback, while slide content supports clarity and trust, with the combined approach offering the strongest mean gains yet potential cognitive load concerns. The work demonstrates that AI-powered, multimodal feedback can match human effectiveness and offers practical implications for integrating such systems into learning platforms, while highlighting needs for better explainability, reliability, and load management to maximize adoption and impact.

Abstract

Feedback is important in supporting student learning. While various automated feedback systems have been implemented to make the feedback scalable, many existing solutions only focus on generating text-based feedback. As is indicated in the multimedia learning principle, learning with more modalities could help utilize more separate channels, reduce the cognitive load and facilitate students' learning. Hence, it is important to explore the potential of Artificial Intelligence (AI) in feedback generation from and to different modalities. Our study leverages Large Language Models (LLMs) for textual feedback with the supplementary guidance from other modality - relevant lecture slide retrieved from the slides hub. Through an online crowdsourcing study (N=91), this study investigates learning gains and student perceptions using a 2x2 design (i.e., human feedback vs. AI feedback and with vs. without relevant slide), evaluating the clarity, engagement, perceived effectiveness, and reliability) of AI-facilitated multimodal feedback. We observed significant pre-to-post learning gains across all conditions. However, the differences in these gains were not statistically significant between conditions. The post-survey revealed that students found the slide feedback helpful in their learning process, though they reported difficulty in understanding it. Regarding the AI-generated open-ended feedback, students considered it personalized and relevant to their responses, but they expressed lower trust in the AI feedback compared to human-generated feedback.

SlideItRight: Using AI to Find Relevant Slides and Provide Feedback for Open-Ended Questions

TL;DR

SlideItRight tackles the challenge of scalable, personalized feedback by combining LLM-generated textual feedback with retrieved, slide-grounded references. The system employs retrieval-augmented generation and a 2×2 crowdsourced study to compare human, AI, slide, and combined feedback modalities, measuring pre/post learning gains and post-survey perceptions. Results show significant learning gains across all conditions but no statistically significant differences between modalities; AI offers strong personalization and rapid feedback, while slide content supports clarity and trust, with the combined approach offering the strongest mean gains yet potential cognitive load concerns. The work demonstrates that AI-powered, multimodal feedback can match human effectiveness and offers practical implications for integrating such systems into learning platforms, while highlighting needs for better explainability, reliability, and load management to maximize adoption and impact.

Abstract

Feedback is important in supporting student learning. While various automated feedback systems have been implemented to make the feedback scalable, many existing solutions only focus on generating text-based feedback. As is indicated in the multimedia learning principle, learning with more modalities could help utilize more separate channels, reduce the cognitive load and facilitate students' learning. Hence, it is important to explore the potential of Artificial Intelligence (AI) in feedback generation from and to different modalities. Our study leverages Large Language Models (LLMs) for textual feedback with the supplementary guidance from other modality - relevant lecture slide retrieved from the slides hub. Through an online crowdsourcing study (N=91), this study investigates learning gains and student perceptions using a 2x2 design (i.e., human feedback vs. AI feedback and with vs. without relevant slide), evaluating the clarity, engagement, perceived effectiveness, and reliability) of AI-facilitated multimodal feedback. We observed significant pre-to-post learning gains across all conditions. However, the differences in these gains were not statistically significant between conditions. The post-survey revealed that students found the slide feedback helpful in their learning process, though they reported difficulty in understanding it. Regarding the AI-generated open-ended feedback, students considered it personalized and relevant to their responses, but they expressed lower trust in the AI feedback compared to human-generated feedback.
Paper Structure (30 sections, 4 figures, 3 tables)

This paper contains 30 sections, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Feedback in Learning
  4. Multimodal Feedback
  5. Large Language Models for Feedback Generation
  6. System Implementation
  7. Multimodal Input
  8. Multimodal Feedback Generation: Text Feedback with Lecture Slides
  9. Personalized Feedback
  10. Efficient Responding Time
  11. Experimental Design
  12. Learning Materials
  13. Conditions
  14. Study Procedure
  15. Participant Recruitment
  16. ...and 15 more sections

Figures (4)

  • Figure 1: SlideItRight Student Interface and Functionality Overview. On the left is (1) the multimodal feedback panel, which includes (a) AI generated feedback on student responses, (b) OpenAI vision's understanding of the retrieved slide page, (c) a slide page related to the preset question, and (d) a zoom in button to enlarge and look through the retrieved slide page. On the right is (2) the user interaction panel, which features: (a) a preset question whose interface is currently minimized for Intelligent Tutoring System (ITS) platform integration, (b) student answer input box, (c) a status indicator of whether the student’s input is successfully cached.
  • Figure 2: SlideItRight Workflow. The system retrieves relevant course slides based on the content of a given question, to enhance multimodal feedback generation for the student responses.
  • Figure 3: The Feedback Format for Four Experimental Conditions
  • Figure 4: Comparison of Learning Gains and Test Scores Across Groups