Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How Learner Control and Explainable Learning Analytics on Skill Mastery Shape Student Desires to Finish and Avoid Loss in Tutored Practice

Conrad Borchers, Jeroen Ooge, Cindy Peng, Vincent Aleven

TL;DR

This study investigates how learner control and explainable learning analytics shape student practice decisions in tutoring systems, focusing on self-regulated homework contexts. It implements a math-practice app with three control modes and what-if mastery explanations, using mastery bars and Bayesian Knowledge Tracing to visualize progress, and evaluates them through semi-structured interviews with six middle school students. Thematic analysis reveals four student needs—desire to improve mastery, finish, avoid mastery losses, and control—that interact with analytics to influence problem selection, with what-if explanations shifting focus toward achievable progress. The findings suggest that explainable analytics can bolster SRL and motivation in homework contexts, though future work should address cognitive load, interpretation, and validation across diverse populations in real-world settings.

Abstract

Personalized problem selection enhances student practice in tutoring systems. Prior research has focused on transparent problem selection that supports learner control but rarely engages learners in selecting practice materials. We explored how different levels of control (i.e., full AI control, shared control, and full learner control), combined with showing learning analytics on skill mastery and visual what-if explanations, can support students in practice contexts requiring high degrees of self-regulation, such as homework. Semi-structured interviews with six middle school students revealed three key insights: (1) participants highly valued learner control for an enhanced learning experience and better self-regulation, especially because most wanted to avoid losses in skill mastery; (2) only seeing their skill mastery estimates often made participants base problem selection on their weaknesses; and (3) what-if explanations stimulated participants to focus more on their strengths and improve skills until they were mastered. These findings show how explainable learning analytics could shape students' selection strategies when they have control over what to practice. They suggest promising avenues for helping students learn to regulate their effort, motivation, and goals during practice with tutoring systems.

How Learner Control and Explainable Learning Analytics on Skill Mastery Shape Student Desires to Finish and Avoid Loss in Tutored Practice

TL;DR

This study investigates how learner control and explainable learning analytics shape student practice decisions in tutoring systems, focusing on self-regulated homework contexts. It implements a math-practice app with three control modes and what-if mastery explanations, using mastery bars and Bayesian Knowledge Tracing to visualize progress, and evaluates them through semi-structured interviews with six middle school students. Thematic analysis reveals four student needs—desire to improve mastery, finish, avoid mastery losses, and control—that interact with analytics to influence problem selection, with what-if explanations shifting focus toward achievable progress. The findings suggest that explainable analytics can bolster SRL and motivation in homework contexts, though future work should address cognitive load, interpretation, and validation across diverse populations in real-world settings.

Abstract

Personalized problem selection enhances student practice in tutoring systems. Prior research has focused on transparent problem selection that supports learner control but rarely engages learners in selecting practice materials. We explored how different levels of control (i.e., full AI control, shared control, and full learner control), combined with showing learning analytics on skill mastery and visual what-if explanations, can support students in practice contexts requiring high degrees of self-regulation, such as homework. Semi-structured interviews with six middle school students revealed three key insights: (1) participants highly valued learner control for an enhanced learning experience and better self-regulation, especially because most wanted to avoid losses in skill mastery; (2) only seeing their skill mastery estimates often made participants base problem selection on their weaknesses; and (3) what-if explanations stimulated participants to focus more on their strengths and improve skills until they were mastered. These findings show how explainable learning analytics could shape students' selection strategies when they have control over what to practice. They suggest promising avenues for helping students learn to regulate their effort, motivation, and goals during practice with tutoring systems.

Paper Structure

This paper contains 14 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Learning Analytics for Practice Support and Self-Regulated Learning
  4. Learner Control in Tutoring Systems
  5. Explainable AI in Education
  6. Methods and Materials
  7. Math Practice App
  8. Participant Recruitment, Procedure, and Data Analysis
  9. Results
  10. Discussion
  11. Students Desire Practice Control, Mainly To Avoid Mastery Losses
  12. Explainable Learning Analytics Can Spur Desires to Improve and Finish Practice
  13. Limitations and Future work
  14. Conclusions

Figures (3)

  • Figure 1: The recommendation algorithm underlying our app is based on skills associated with math problems. We designed three granularity levels of control: full AI control, shared control, and full learner control.
  • Figure 2: Our six designs provide one of three control mechanisms and visual what-if explanations.
  • Figure 3: Summary of the four main desires participants expressed during the interviews and how our system supports them.