Designing Visual Explanations and Learner Controls to Engage Adolescents in AI-Supported Exercise Selection

TL;DR

The paper addresses transparency and controllability in AI-supported education for adolescents and presents a four-stage, human-centered design process to integrate what-if explanations and learner control into AI-recommended exercise sequences. It demonstrates that what-if explanations, coupled with a pre-practice difficulty slider and motivational feedback, can boost motivation and cognitive engagement, while why explanations are more useful for teachers than students. The work provides design insights and guidelines for combining transparency with learner agency in educational AI, and it highlights the need for adaptive explanations and guardrails when enabling learner control. The findings lay groundwork for subsequent large-scale, quantitative studies to quantify impacts on metacognition, motivation, trust, and learning outcomes in real-classroom deployments.

Abstract

E-learning platforms that personalise content selection with AI are often criticised for lacking transparency and controllability. Researchers have therefore proposed solutions such as open learner models and letting learners select from ranked recommendations, which engage learners before or after the AI-supported selection process. However, little research has explored how learners - especially adolescents - could engage during such AI-supported decision-making. To address this open challenge, we iteratively designed and implemented a control mechanism that enables learners to steer the difficulty of AI-compiled exercise series before practice, while interactively analysing their control's impact in a 'what-if' visualisation. We evaluated our prototypes through four qualitative studies involving adolescents, teachers, EdTech professionals, and pedagogical experts, focusing on different types of visual explanations for recommendations. Our findings suggest that 'why' explanations do not always meet the explainability needs of young learners but can benefit teachers. Additionally, 'what-if' explanations were well-received for their potential to boost motivation. Overall, our work illustrates how combining learner control and visual explanations can be operationalised on e-learning platforms for adolescents. Future research can build upon our designs for 'why' and 'what-if' explanations and verify our preliminary findings.

Figures (5)

• Figure 1: Overview of the four user studies we conducted with adolescents, teachers, EdTech professionals, and pedagogical experts, together with the goals of each study. Larger screenshots of our interfaces are presented in Figures \ref{['fig:prototype1']} to \ref{['fig:flow']}.
• Figure 2: Screenshot of our first prototype. Left: the recommended exercise series. Right: an explanation panel shows how previous attempts established learners' mastery levels and why and justifies recommendations with what-if explanations.
• Figure 3: Screenshots of our second prototype with redesigned why and what-if explanations. Hovering scatter plots adds vertical jitter, and the info button in the mastery level labels reveals previous performance.
• Figure 4: Screenshots of our third prototype with a slider for learner control, linked to a redesigned what-if explanation (left) or motivational feedback (right). The latter two components are interactively updated based on the chosen slider value.
• Figure 5: Workflow on our e-learning platform: Learners choose a topic and inform themselves of the mastery level system; The platform compiles a series of three exercises that fit learners' mastery level. Learners can steer its difficulty while seeing a what-if explanation or motivational feedback; Learners complete the exercises and can restart the practice cycle.