Differentiating Student Feedbacks for Knowledge Tracing

TL;DR

Knowledge tracing models often inflate accuracy by focusing on low-discrimination responses, neglecting high-discrimination cases that truly reveal student mastery. The authors propose DR4KT, a discrimination-aware framework comprising a frequency-aware correctness tendency estimator, loss reweighting for discriminative responses, and an adaptive predictive score fuser to balance predictions. Through pretraining and joint training, DR4KT improves the performance of three KT backbones (DKT, SAKT, AKT) across three public datasets, while maintaining robust tracing of students' knowledge states on high-discrimination items. The approach addresses the discrimination imbalance, yielding more reliable knowledge tracing and actionable insights for personalized education. All components are designed to be model-agnostic, enabling straightforward integration into existing KT systems and improving their practical impact.

Abstract

Knowledge tracing (KT) is a crucial task in computer-aided education and intelligent tutoring systems, predicting students' performance on new questions from their responses to prior ones. An accurate KT model can capture a student's mastery level of different knowledge topics, as reflected in their predicted performance on different questions. This helps improve the learning efficiency by suggesting appropriate new questions that complement students' knowledge states. However, current KT models have significant drawbacks that they neglect the imbalanced discrimination of historical responses. A significant proportion of question responses provide limited information for discerning students' knowledge mastery, such as those that demonstrate uniform performance across different students. Optimizing the prediction of these cases may increase overall KT accuracy, but also negatively impact the model's ability to trace personalized knowledge states, especially causing a deceptive surge of performance. Towards this end, we propose a framework to reweight the contribution of different responses based on their discrimination in training. Additionally, we introduce an adaptive predictive score fusion technique to maintain accuracy on less discriminative responses, achieving proper balance between student knowledge mastery and question difficulty. Experimental results demonstrate that our framework enhances the performance of three mainstream KT methods on three widely-used datasets.

Figures (5)

• Figure 1: Two examples of questions answered by students in the Eedi dataset. The accuracy of four KT methods in predicting the responses to these questions is shown on the left. "Disc." means discrimination. $r=1$ indicates a correct response.
• Figure 2: Response proportions and prediction accuracy of four typical KT methods at different discrimination levels of three datasets. The percentage pairs indicate the proportions of low discriminative responses (from 0 to 0.5) and high ones (from 0.5 to 1).
• Figure 3: The entire framework of DR4KT. The question correctness tendency estimator is pretrained in advance. The dashed arrows indicate the inference only in training.
• Figure 4: Hyper-parameter analysis of DR4KT on ASSIST09.
• Figure 5: An example of DKT with and without DR4KT tracing student knowledge mastery. We choose a student's response sequence from the Eedi dataset. The gray squares indicate the calculated discrimination scores of each response. Each green square denotes the student's updated knowledge mastery of the corresponding knowledge concept after completing the response. Solid circles represent correct responses and hollow ones represent incorrect responses.