CIKT: A Collaborative and Iterative Knowledge Tracing Framework with Large Language Models
Runze Li, Siyu Wu, Jun Wang, Wei Zhang
TL;DR
CIKT addresses explainability and scalability in knowledge tracing by introducing an Analyst that generates structured, interpretable student profiles and a Predictor that forecasts outcomes conditioned on these profiles within an iterative Kahneman-Tversky optimization loop. It distills profiling knowledge from a teacher LLM, propagates this through Profiling and Reasoning stages, and continuously improves via Iteration, where profile quality is refined to boost predictive accuracy for binary outcomes $y_{s,t} \in \{0,1\}$. Across three public datasets, CIKT achieves competitive or superior accuracy and F1 scores compared to traditional KT baselines and direct LLM approaches, with pronounced benefits on long histories. The framework offers a practical path toward explainable, self-improving KT systems suitable for scalable deployment in educational settings.
Abstract
Knowledge Tracing (KT) aims to model a student's learning state over time and predict their future performance. However, traditional KT methods often face challenges in explainability, scalability, and effective modeling of complex knowledge dependencies. While Large Language Models (LLMs) present new avenues for KT, their direct application often struggles with generating structured, explainable student representations and lacks mechanisms for continuous, task-specific refinement. To address these gaps, we propose Collaborative Iterative Knowledge Tracing (CIKT), a framework that harnesses LLMs to enhance both prediction accuracy and explainability. CIKT employs a dual-component architecture: an Analyst generates dynamic, explainable user profiles from student historical responses, and a Predictor utilizes these profiles to forecast future performance. The core of CIKT is a synergistic optimization loop. In this loop, the Analyst is iteratively refined based on the predictive accuracy of the Predictor, which conditions on the generated profiles, and the Predictor is subsequently retrained using these enhanced profiles. Evaluated on multiple educational datasets, CIKT demonstrates significant improvements in prediction accuracy, offers enhanced explainability through its dynamically updated user profiles, and exhibits improved scalability. Our work presents a robust and explainable solution for advancing knowledge tracing systems, effectively bridging the gap between predictive performance and model transparency.