Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HISE-KT: Synergizing Heterogeneous Information Networks and LLMs for Explainable Knowledge Tracing with Meta-Path Optimization

Zhiyi Duan, Zixing Shi, Hongyu Yuan, Qi Wang

TL;DR

HISE-KT tackles the dual challenges of noisy meta-paths in HIN-based KT and limited interpretability in LLM-driven KT by integrating a multi-relational heterogenous information network with LLM-based meta-path scoring and cross-student retrieval. The framework constructs an MRHIN with five node types, uses an LLM to score and prune meta-path instances, retrieves similar students via rule-based collaborative filtering, and generates zero-shot KT predictions with structured prompts that include explanations. Empirical results on four public datasets show superior predictive performance and enhanced explainability versus state-of-the-art baselines, with ablations confirming the importance of meta-path quality assessment and cross-student context. The approach provides a principled, transparent, and scalable paradigm for explainable KT and educational AI.

Abstract

Knowledge Tracing (KT) aims to mine students' evolving knowledge states and predict their future question-answering performance. Existing methods based on heterogeneous information networks (HINs) are prone to introducing noises due to manual or random selection of meta-paths and lack necessary quality assessment of meta-path instances. Conversely, recent large language models (LLMs)-based methods ignore the rich information across students, and both paradigms struggle to deliver consistently accurate and evidence-based explanations. To address these issues, we propose an innovative framework, HIN-LLM Synergistic Enhanced Knowledge Tracing (HISE-KT), which seamlessly integrates HINs with LLMs. HISE-KT first builds a multi-relationship HIN containing diverse node types to capture the structural relations through multiple meta-paths. The LLM is then employed to intelligently score and filter meta-path instances and retain high-quality paths, pioneering automated meta-path quality assessment. Inspired by educational psychology principles, a similar student retrieval mechanism based on meta-paths is designed to provide a more valuable context for prediction. Finally, HISE-KT uses a structured prompt to integrate the target student's history with the retrieved similar trajectories, enabling the LLM to generate not only accurate predictions but also evidence-backed, explainable analysis reports. Experiments on four public datasets show that HISE-KT outperforms existing KT baselines in both prediction performance and interpretability.

HISE-KT: Synergizing Heterogeneous Information Networks and LLMs for Explainable Knowledge Tracing with Meta-Path Optimization

TL;DR

HISE-KT tackles the dual challenges of noisy meta-paths in HIN-based KT and limited interpretability in LLM-driven KT by integrating a multi-relational heterogenous information network with LLM-based meta-path scoring and cross-student retrieval. The framework constructs an MRHIN with five node types, uses an LLM to score and prune meta-path instances, retrieves similar students via rule-based collaborative filtering, and generates zero-shot KT predictions with structured prompts that include explanations. Empirical results on four public datasets show superior predictive performance and enhanced explainability versus state-of-the-art baselines, with ablations confirming the importance of meta-path quality assessment and cross-student context. The approach provides a principled, transparent, and scalable paradigm for explainable KT and educational AI.

Abstract

Knowledge Tracing (KT) aims to mine students' evolving knowledge states and predict their future question-answering performance. Existing methods based on heterogeneous information networks (HINs) are prone to introducing noises due to manual or random selection of meta-paths and lack necessary quality assessment of meta-path instances. Conversely, recent large language models (LLMs)-based methods ignore the rich information across students, and both paradigms struggle to deliver consistently accurate and evidence-based explanations. To address these issues, we propose an innovative framework, HIN-LLM Synergistic Enhanced Knowledge Tracing (HISE-KT), which seamlessly integrates HINs with LLMs. HISE-KT first builds a multi-relationship HIN containing diverse node types to capture the structural relations through multiple meta-paths. The LLM is then employed to intelligently score and filter meta-path instances and retain high-quality paths, pioneering automated meta-path quality assessment. Inspired by educational psychology principles, a similar student retrieval mechanism based on meta-paths is designed to provide a more valuable context for prediction. Finally, HISE-KT uses a structured prompt to integrate the target student's history with the retrieved similar trajectories, enabling the LLM to generate not only accurate predictions but also evidence-backed, explainable analysis reports. Experiments on four public datasets show that HISE-KT outperforms existing KT baselines in both prediction performance and interpretability.

Paper Structure

This paper contains 32 sections, 14 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. HIN-Based KT Methods
  4. LLM-Based KT Methods
  5. Methodology
  6. Multi-Relationship Heterogeneous Information Network Construction Module
  7. LLM-Powered Meta-path Optimization Module
  8. Rule-Based Collaborative Filtering Module
  9. LLM Prediction Module
  10. Experiment
  11. Experimental Settings
  12. Main Results
  13. Ablation Study
  14. Parameter Sensitivity
  15. Case Study
  16. ...and 17 more sections

Figures (7)

  • Figure 1: The framework of HISE-KT: (1) Build a multi-relationship HIN, then define and instantiate multiple meta-paths; (2) Employ the LLM to score and select high-quality meta-path instances; (3) Retrieve the target student's similar peers through high-quality meta-path; (4) Input cross-student information into LLM for KT prediction and analysis reports generation.
  • Figure 2: Parameter‐sensitivity analysis.
  • Figure 3: Case study of a target student $U_{4067}$ on question $Q_{3352}$.
  • Figure 4: Prompt of meta-path scoring (1)
  • Figure 5: Prompt of meta-path scoring (2)
  • ...and 2 more figures