Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CLST: Cold-Start Mitigation in Knowledge Tracing by Aligning a Generative Language Model as a Students' Knowledge Tracer

Heeseok Jung, Jaesang Yoo, Yohaan Yoon, Yeonju Jang

TL;DR

This work tackles cold-start knowledge tracing by moving beyond ID-based approaches and leveraging generative LLMs. The authors introduce CLST, which formats KT as a language processing task (KTLP) and fine-tunes a generative LLM with LoRA on KT-formatted data across math, social studies, and science, achieving superior predictive accuracy and calibration in data-scarce settings. They demonstrate strong cross-domain generalization and show that description-based exercise representations better leverage KC relationships encoded in LLMs. The findings suggest practical pathways for deploying robust KT in early-stage ITS deployments and across domains, with implications for scalability and personalized learning.

Abstract

Knowledge tracing (KT), wherein students' problem-solving histories are used to estimate their current levels of knowledge, has attracted significant interest from researchers. However, most existing KT models were developed with an ID-based paradigm, which exhibits limitations in cold-start performance. These limitations can be mitigated by leveraging the vast quantities of external knowledge possessed by generative large language models (LLMs). In this study, we propose cold-start mitigation in knowledge tracing by aligning a generative language model as a students' knowledge tracer (CLST) as a framework that utilizes a generative LLM as a knowledge tracer. Upon collecting data from math, social studies, and science subjects, we framed the KT task as a natural language processing task, wherein problem-solving data are expressed in natural language, and fine-tuned the generative LLM using the formatted KT dataset. Subsequently, we evaluated the performance of the CLST in situations of data scarcity using various baseline models for comparison. The results indicate that the CLST significantly enhanced performance with a dataset of fewer than 100 students in terms of prediction, reliability, and cross-domain generalization.

CLST: Cold-Start Mitigation in Knowledge Tracing by Aligning a Generative Language Model as a Students' Knowledge Tracer

TL;DR

This work tackles cold-start knowledge tracing by moving beyond ID-based approaches and leveraging generative LLMs. The authors introduce CLST, which formats KT as a language processing task (KTLP) and fine-tunes a generative LLM with LoRA on KT-formatted data across math, social studies, and science, achieving superior predictive accuracy and calibration in data-scarce settings. They demonstrate strong cross-domain generalization and show that description-based exercise representations better leverage KC relationships encoded in LLMs. The findings suggest practical pathways for deploying robust KT in early-stage ITS deployments and across domains, with implications for scalability and personalized learning.

Abstract

Knowledge tracing (KT), wherein students' problem-solving histories are used to estimate their current levels of knowledge, has attracted significant interest from researchers. However, most existing KT models were developed with an ID-based paradigm, which exhibits limitations in cold-start performance. These limitations can be mitigated by leveraging the vast quantities of external knowledge possessed by generative large language models (LLMs). In this study, we propose cold-start mitigation in knowledge tracing by aligning a generative language model as a students' knowledge tracer (CLST) as a framework that utilizes a generative LLM as a knowledge tracer. Upon collecting data from math, social studies, and science subjects, we framed the KT task as a natural language processing task, wherein problem-solving data are expressed in natural language, and fine-tuned the generative LLM using the formatted KT dataset. Subsequently, we evaluated the performance of the CLST in situations of data scarcity using various baseline models for comparison. The results indicate that the CLST significantly enhanced performance with a dataset of fewer than 100 students in terms of prediction, reliability, and cross-domain generalization.
Paper Structure (21 sections, 8 equations, 9 figures, 3 tables)

This paper contains 21 sections, 8 equations, 9 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. 2.1 Knowledge tracing
  4. 2.2 Generative LLMs in personalized learning
  5. Methodology
  6. 3.1 Problem formulation
  7. 3.2 CLST: Cold-start mitigation in KT by aligning a generative language model as a students’ knowledge tracer
  8. 3.2.1 KTLP formatting: Utilizing natural language as KT information carrier
  9. 3.2.2 Response prediction with CLST
  10. 3.2.3 Instruction fine-tuning using KTLP-formatted data with LoRA architecture
  11. 3.3 Dataset
  12. 3.4 Baseline models
  13. 3.5 Experimental settings
  14. Results and discussion
  15. 4.1 Predictive performance under cold-start scenarios
  16. ...and 6 more sections

Figures (9)

  • Figure 1: Sample KTLP-formatted interaction.
  • Figure 2: Cold-start performance on each dataset.
  • Figure 3: Examples of the two methods for representing exercises.
  • Figure 4: Changes in output calibration as an effect of fine-tuning.
  • Figure 5: Learning trajectories of student 'A' in mathematics
  • ...and 4 more figures