Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DiaCDM: Cognitive Diagnosis in Teacher-Student Dialogues using the Initiation-Response-Evaluation Framework

Rui Jia, Yuang Wei, Ruijia Li, Yuan-Hao Jiang, Xinyu Xie, Yaomin Shen, Min Zhang, Bo Jiang

TL;DR

This work tackles cognitive diagnosis in real-world teacher–student dialogues, where dynamic and unstructured language challenges static, test-based CDMs. It introduces DiaCDM, a framework that uses an Initiation–Response–Evaluation (IRE) structure and AMR-based graph encoding to extract diagnostic semantics from teacher questions and student responses, supplemented by three cognitive-state channels (C_q, C_t, C_s) that are fused into a final mastery representation $h_c$. The method integrates DINA/IRT-inspired prediction with AMR-guided question encoding and a multidimensional state model to predict future performance while improving interpretability. Evaluations on three real-world dialogue datasets demonstrate superior diagnostic accuracy and provide actionable interpretability insights for instructional practice, with code released for replication.

Abstract

While cognitive diagnosis (CD) effectively assesses students' knowledge mastery from structured test data, applying it to real-world teacher-student dialogues presents two fundamental challenges. Traditional CD models lack a suitable framework for handling dynamic, unstructured dialogues, and it's difficult to accurately extract diagnostic semantics from lengthy dialogues. To overcome these hurdles, we propose DiaCDM, an innovative model. We've adapted the initiation-response-evaluation (IRE) framework from educational theory to design a diagnostic framework tailored for dialogue. We also developed a unique graph-based encoding method that integrates teacher questions with relevant knowledge components to capture key information more precisely. To our knowledge, this is the first exploration of cognitive diagnosis in a dialogue setting. Experiments on three real-world dialogue datasets confirm that DiaCDM not only significantly improves diagnostic accuracy but also enhances the results' interpretability, providing teachers with a powerful tool for assessing students' cognitive states. The code is available at https://github.com/Mind-Lab-ECNU/DiaCDM/tree/main.

DiaCDM: Cognitive Diagnosis in Teacher-Student Dialogues using the Initiation-Response-Evaluation Framework

TL;DR

This work tackles cognitive diagnosis in real-world teacher–student dialogues, where dynamic and unstructured language challenges static, test-based CDMs. It introduces DiaCDM, a framework that uses an Initiation–Response–Evaluation (IRE) structure and AMR-based graph encoding to extract diagnostic semantics from teacher questions and student responses, supplemented by three cognitive-state channels (C_q, C_t, C_s) that are fused into a final mastery representation . The method integrates DINA/IRT-inspired prediction with AMR-guided question encoding and a multidimensional state model to predict future performance while improving interpretability. Evaluations on three real-world dialogue datasets demonstrate superior diagnostic accuracy and provide actionable interpretability insights for instructional practice, with code released for replication.

Abstract

While cognitive diagnosis (CD) effectively assesses students' knowledge mastery from structured test data, applying it to real-world teacher-student dialogues presents two fundamental challenges. Traditional CD models lack a suitable framework for handling dynamic, unstructured dialogues, and it's difficult to accurately extract diagnostic semantics from lengthy dialogues. To overcome these hurdles, we propose DiaCDM, an innovative model. We've adapted the initiation-response-evaluation (IRE) framework from educational theory to design a diagnostic framework tailored for dialogue. We also developed a unique graph-based encoding method that integrates teacher questions with relevant knowledge components to capture key information more precisely. To our knowledge, this is the first exploration of cognitive diagnosis in a dialogue setting. Experiments on three real-world dialogue datasets confirm that DiaCDM not only significantly improves diagnostic accuracy but also enhances the results' interpretability, providing teachers with a powerful tool for assessing students' cognitive states. The code is available at https://github.com/Mind-Lab-ECNU/DiaCDM/tree/main.

Paper Structure

This paper contains 9 sections, 18 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. related work
  3. Preliminary
  4. Method
  5. Dialogue Encoding Module
  6. Cognitive State Modeling Module
  7. Prediction Module
  8. Experiments
  9. Conclusion

Figures (3)

  • Figure 1: (a) Cognitive diagnosis: Diagnose students’ cognitive level of knowledge components through structured test items. (b) Dialogue-based CD: Dynamically identify students’ cognition of knowledge components by analyzing teacher-student dialogues.
  • Figure 2: Overview of DiaCDM: (a) Encode the teacher's questions, student responses, and teacher evaluations separately. (b) Capture the matching of the teacher's questions, the cognitive state in student responses, and the cognitive state in teacher evaluations from the encoded vectors. (c) Predict the student's future performance by integrating the information from these three dimensions with educational theory.
  • Figure 3: Visualization of Main and Sub-feature Dynamics in Cognitive State Identification.