Explainable Few-shot Knowledge Tracing

TL;DR

The paper addresses the misalignment between traditional knowledge tracing and real-world teaching by introducing Explainable Few-shot Knowledge Tracing, which leverages large language models to infer student knowledge from a small number of responses and generate natural language explanations. It proposes a cognition-guided framework with Observation, Cognition, and Interpretation modules that use in-context learning and reasoning to produce both predictions and interpretable explanations. Across three public KT datasets, the approach achieves comparable or superior accuracy and F1 scores to strong baselines, with GPT-4 and GLM4 often outperforming, especially in data-rich modes; smaller models like GLM3-6B underperform due to context and instruction-following limitations. The work highlights practical benefits for education, including flexible adaptation to teaching contexts and the potential for richer feedback, while noting costs, privacy, and bias considerations that warrant further research. Overall, this framework expands KT from purely numerical accuracy to actionable, explainable diagnostics suitable for real-world classroom use.

Abstract

Knowledge tracing (KT), aiming to mine students' mastery of knowledge by their exercise records and predict their performance on future test questions, is a critical task in educational assessment. While researchers achieved tremendous success with the rapid development of deep learning techniques, current knowledge tracing tasks fall into the cracks from real-world teaching scenarios. Relying heavily on extensive student data and solely predicting numerical performances differs from the settings where teachers assess students' knowledge state from limited practices and provide explanatory feedback. To fill this gap, we explore a new task formulation: Explainable Few-shot Knowledge Tracing. By leveraging the powerful reasoning and generation abilities of large language models (LLMs), we then propose a cognition-guided framework that can track the student knowledge from a few student records while providing natural language explanations. Experimental results from three widely used datasets show that LLMs can perform comparable or superior to competitive deep knowledge tracing methods. We also discuss potential directions and call for future improvements in relevant topics.

Figures (16)

• Figure 1: Conventional knowledge tracing and explainable few-shot knowledge tracing
• Figure 2: The cognition-guided framework for explainable few-shot knowledge tracing
• Figure 3: Dataset of different modes.
• Figure 4: Case study of GLM3-6B
• Figure 5: Case study of GLM4 and GPT-4