Enhancing Knowledge Tracing with Concept Map and Response Disentanglement

TL;DR

The paper addresses the limited expressiveness of traditional KT models that rely solely on binary correctness by leveraging MCQ option choices, unchosen responses, and a concept-map structure to more accurately track knowledge states. CRKT introduces a disentangled response encoder, a temporal-cumulative knowledge retriever, a concept-map encoder with a question-specific edge weighting, and an IRT-inspired prediction mechanism, trained with KT, top-K relevance, and contrastive losses. Empirical results across five diverse datasets show CRKT achieving superior ACC and AUC compared to strong baselines, with ablations underscoring the value of option responses and concept maps, and case studies illustrating interpretability. The approach promises actionable, personalized feedback in real-world education and motivates future work on dynamic concept relationships and unsupervised concept mapping.

Abstract

In the rapidly advancing realm of educational technology, it becomes critical to accurately trace and understand student knowledge states. Conventional Knowledge Tracing (KT) models have mainly focused on binary responses (i.e., correct and incorrect answers) to questions. Unfortunately, they largely overlook the essential information in students' actual answer choices, particularly for Multiple Choice Questions (MCQs), which could help reveal each learner's misconceptions or knowledge gaps. To tackle these challenges, we propose the Concept map-driven Response disentanglement method for enhancing Knowledge Tracing (CRKT) model. CRKT benefits KT by directly leveraging answer choices--beyond merely identifying correct or incorrect answers--to distinguish responses with different incorrect choices. We further introduce the novel use of unchosen responses by employing disentangled representations to get insights from options not selected by students. Additionally, CRKT tracks the student's knowledge state at the concept level and encodes the concept map, representing the relationships between them, to better predict unseen concepts. This approach is expected to provide actionable feedback, improving the learning experience. Our comprehensive experiments across multiple datasets demonstrate CRKT's effectiveness, achieving superior performance in prediction accuracy and interpretability over state-of-the-art models.

Figures (6)

• Figure 1: Limitations of conventional KT models: (a) Reliance on binary responses only and (b) Unutilized concept relationships
• Figure 2: Examples of disentangled response representation for the question in Fig. \ref{['fig:intro']}.
• Figure 3: The architecture of Concept map-driven Response disentanglement method for enhancing Knowledge Tracing model (CRKT).
• Figure 4: The experimental results in ablation study.
• Figure 5: Comparison of accuracy between DTransformer, DP-DKT, and CRKT across different average question correctness rates(x-axis) and interaction rate(bar).