Disentangled Knowledge Tracing for Alleviating Cognitive Bias

TL;DR

DisKT tackles cognitive bias in Knowledge Tracing by explicitly modeling causal effects and disentangling student abilities into familiar and unfamiliar components. It introduces Rasch-based embeddings, a Transformer-based Knowledge Extractor, a Contradictory Attention mechanism to guard against guessing/mistaking, and a variant of Item Response Theory for interpretability. Empirical results across 11 public datasets and three bias-strength simulations show that DisKT consistently improves accuracy while significantly reducing bias compared to 16 baselines. The work provides a principled, interpretable framework for debiasing KT with potential impact on ITS personalization and exercise recommendations.

Abstract

In the realm of Intelligent Tutoring System (ITS), the accurate assessment of students' knowledge states through Knowledge Tracing (KT) is crucial for personalized learning. However, due to data bias, $\textit{i.e.}$, the unbalanced distribution of question groups ($\textit{e.g.}$, concepts), conventional KT models are plagued by cognitive bias, which tends to result in cognitive underload for overperformers and cognitive overload for underperformers. More seriously, this bias is amplified with the exercise recommendations by ITS. After delving into the causal relations in the KT models, we identify the main cause as the confounder effect of students' historical correct rate distribution over question groups on the student representation and prediction score. Towards this end, we propose a Disentangled Knowledge Tracing (DisKT) model, which separately models students' familiar and unfamiliar abilities based on causal effects and eliminates the impact of the confounder in student representation within the model. Additionally, to shield the contradictory psychology ($\textit{e.g.}$, guessing and mistaking) in the students' biased data, DisKT introduces a contradiction attention mechanism. Furthermore, DisKT enhances the interpretability of the model predictions by integrating a variant of Item Response Theory. Experimental results on 11 benchmarks and 3 synthesized datasets with different bias strengths demonstrate that DisKT significantly alleviates cognitive bias and outperforms 16 baselines in evaluation accuracy.

Figures (6)

• Figure 1: Illustration of cognitive bias.
• Figure 2: The causal graphs for conventional and counterfactual KT. $\ast$ denotes the reference values.
• Figure 3: The architecture of the Disentangled Knowledge Tracing model (DisKT).
• Figure 4: The performance comparison between the competitive baselines and DisKT on alleviating cognitive bias. (left): AUC performance changes of DisKT and baselines optimized with different bias strengths on ednet, tested with ednet-low and ednet-high respectively. (right): $E_{KL}$ scores of several representative models on three datasets with different bias strengths.
• Figure 5: Empirical evidence of guessing and mistaking verified by DKT applied to the assist09 dataset.