Experimental Interface for Multimodal and Large Language Model Based Explanations of Educational Recommender Systems
Hasan Abu-Rasheed, Christian Weber, Madjid Fathi
TL;DR
The paper tackles the need for evidence-based evaluation of explanations in AI-based educational recommendations. It presents an experimental, web-based interface that delivers multimodal and LLM-derived explanations, with modular components for textual, visual, and conversational guidance, informed by pedagogical requirements. The authors describe a two-stage evaluation plan, reporting initial positive feedback from learners and outlining plans to involve educators and broaden LLM support. This work contributes a practical framework for researchers and educators to study the impact of different explainability modalities and their hybrids on learning outcomes and decision-making in educational recommender systems. The approach has practical implications for designing explainability tools that are adaptable, testable, and scalable across diverse educational contexts.
Abstract
In the age of artificial intelligence (AI), providing learners with suitable and sufficient explanations of AI-based recommendation algorithm's output becomes essential to enable them to make an informed decision about it. However, the rapid development of AI approaches for educational recommendations and their explainability is not accompanied by an equal level of evidence-based experimentation to evaluate the learning effect of those explanations. To address this issue, we propose an experimental web-based tool for evaluating multimodal and large language model (LLM) based explainability approaches. Our tool provides a comprehensive set of modular, interactive, and customizable explainability elements, which researchers and educators can utilize to study the role of individual and hybrid explainability methods. We design a two-stage evaluation of the proposed tool, with learners and with educators. Our preliminary results from the first stage show high acceptance of the tool's components, user-friendliness, and an induced motivation to use the explanations for exploring more information about the recommendation.