KTCF: Actionable Recourse in Knowledge Tracing via Counterfactual Explanations for Education

TL;DR

This paper presents KTCF, a counterfactual explanation framework for Knowledge Tracing that produces actionable recourses by incorporating a knowledge-concept (KC) relationship graph. A local, post-hoc optimization tunes counterfactual student responses to flip a KT prediction on a target KC, with a loss that combines prediction accuracy, sparsity, and KC-relational penalties, and then transforms the result into a sequential set of educational instructions via a KC-based path construction. Experiments on the large XES3G5M dataset show that KTCF consistently outperforms baselines in validity and actionability while reducing study burden through concise, coherent KC sequences; certain initializations (soft relaxation and Gaussian noise) yield particularly robust performance. The approach aligns with Bloom's Mastery Learning by linking interpretable, causal explanations to targeted remediation, enabling more responsible and stakeholder-centered AI in education, and pointing to future work with user studies and LLM-enabled instruction generation.

Abstract

Using Artificial Intelligence to improve teaching and learning benefits greater adaptivity and scalability in education. Knowledge Tracing (KT) is recognized for student modeling task due to its superior performance and application potential in education. To this end, we conceptualize and investigate counterfactual explanation as the connection from XAI for KT to education. Counterfactual explanations offer actionable recourse, are inherently causal and local, and easy for educational stakeholders to understand who are often non-experts. We propose KTCF, a counterfactual explanation generation method for KT that accounts for knowledge concept relationships, and a post-processing scheme that converts a counterfactual explanation into a sequence of educational instructions. We experiment on a large-scale educational dataset and show our KTCF method achieves superior and robust performance over existing methods, with improvements ranging from 5.7% to 34% across metrics. Additionally, we provide a qualitative evaluation of our post-processing scheme, demonstrating that the resulting educational instructions help in reducing large study burden. We show that counterfactuals have the potential to advance the responsible and practical use of AI in education. Future works on XAI for KT may benefit from educationally grounded conceptualization and developing stakeholder-centered methods.

Figures (3)

• Figure 1: An application scenario of our counterfactual explanation for KT under an educational context.
• Figure 2: An illustration of our proposed KTCF method for generating counterfactual explanations for KT.
• Figure 3: Ablation study on the sensitivity of KTCF, Wachter, and DiCE methods' performance across hyperparameter variations of initialization strategies -rn and -gs, and across hyperparameter variations of KC Loss $\lambda_{\text{kc}}$.