Personalized Multimodal Feedback Using Multiple External Representations: Strategy Profiles and Learning in High School Physics

TL;DR

The paper addresses how personalized feedback that leverages multiple external representations (MERs) can support physics learning in high school. It employs a 16–24 week observational study with 661 students using a MER-enabled platform and analyzes learning outcomes via linear mixed-effects models, clustering of feedback usage patterns, and ANCOVA-based group comparisons. Key findings show a small but consistent positive association between elaborated MER feedback and post-test performance, with learner representational competence moderating the benefits: less proficient students gain more from alternating verbal, graphical, and mathematical feedback, while highly competent students derive little or no extra gain from additional formats. The work advances personalized instruction in physics by suggesting adaptive feedback designs and informing intelligent tutoring systems, including potential integration with multimodal large language models to tailor feedback delivery.

Abstract

Multiple external representations (MERs) and personalized feedback support physics learning, yet evidence on how personalized feedback can effectively integrate MERs remains limited. This question is particularly timely given the emergence of multimodal large language models. We conducted a 16-24 week observational study in high school physics (N=661) using a computer-based platform that provided verification and optional elaborated feedback in verbal, graphical and mathematical forms. Linear mixed-effects models and strategy-cluster analyses (ANCOVA-adjusted comparisons) tested associations between feedback use and post-test performance and moderation by representational competence. Elaborated multirepresentational feedback showed a small but consistent positive association with post-test scores independent of prior knowledge and confidence. Learners adopted distinct representation-selection strategies; among students with lower representational competence, using a diverse set of representations related to higher learning, whereas this advantage diminished as competence increased. These findings motivate adaptive feedback designs and inform intelligent tutoring systems capable of tailoring feedback elaboration and representational format to learner profiles, advancing personalized instruction in physics education.

Figures (15)

• Figure 1: Examples of physics exercises and automated feedback in the KI4SCool platform. a) Example of a physics exercise in the platform KI4SCool, including the problem statement and the multiple-choice response options. b) Example of feedback displayed to students immediately after selecting an answer, including verification feedback and the option to get additional guidance through three different formats of elaborated feedback.
• Figure 2: Example of verification and elaborated feedback in the three representational formats. Each format is displayed to the student only upon selection.
• Figure 3: Student clusters generated using K-Means algorithm for Cohort 1 (left) and Cohort 2 (right). Each dot represents a student according to the overall verbal, graphical, and mathematical frequency of feedback selected by the student. The clusters are differentiated by colors and their centroid is represented by the cluster labels $0, 1, 2, 3$.
• Figure 4: Student clusters generated using K-Means algorithm for Cohort 1 and Cohort 2 combined. Each dot represents a student according to the overall verbal, graphical, and mathematical frequency of feedback selected by each student. The clusters are differentiated by colors and their centroid is tagged using labels $0, 1, 2, 3$.
• Figure 5: A heatmap representation of the hierarchical clustering of students according to their time-resolved MER-patterns. The color scale from 0 to 7 corresponds to the eight possible feedback combinations outlined in the main text (Section \ref{['subsection: preprocessing']}), and the color scale from 1 to 4 represents the four clusters found. In the horizontal dimension we represent the feedback combination per exercises over time, creating a feedback sequence for each student, vertically stacked.