Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards the Identifiability and Explainability for Personalized Learner Modeling: An Inductive Paradigm

Jiatong Li, Qi Liu, Fei Wang, Jiayu Liu, Zhenya Huang, Fangzhou Yao, Linbo Zhu, Yu Su

TL;DR

An identifiable cognitive diagnosis framework based on a novel response-proficiency-response paradigm inspired by encoder-decoder models is proposed, which leverages inductive learning to eliminate randomness in optimization to guarantee identifiability and captures the monotonicity between overall response data distribution and cognitive states to prevent explainability overfitting.

Abstract

Personalized learner modeling using cognitive diagnosis (CD), which aims to model learners' cognitive states by diagnosing learner traits from behavioral data, is a fundamental yet significant task in many web learning services. Existing cognitive diagnosis models (CDMs) follow the proficiency-response paradigm that views learner traits and question parameters as trainable embeddings and learns them through learner performance prediction. However, we notice that this paradigm leads to the inevitable non-identifiability and explainability overfitting problem, which is harmful to the quantification of learners' cognitive states and the quality of web learning services. To address these problems, we propose an identifiable cognitive diagnosis framework (ID-CDF) based on a novel response-proficiency-response paradigm inspired by encoder-decoder models. Specifically, we first devise the diagnostic module of ID-CDF, which leverages inductive learning to eliminate randomness in optimization to guarantee identifiability and captures the monotonicity between overall response data distribution and cognitive states to prevent explainability overfitting. Next, we propose a flexible predictive module for ID-CDF to ensure diagnosis preciseness. We further present an implementation of ID-CDF, i.e., ID-CDM, to illustrate its usability. Extensive experiments on four real-world datasets with different characteristics demonstrate that ID-CDF can effectively address the problems without loss of diagnosis preciseness.

Towards the Identifiability and Explainability for Personalized Learner Modeling: An Inductive Paradigm

TL;DR

An identifiable cognitive diagnosis framework based on a novel response-proficiency-response paradigm inspired by encoder-decoder models is proposed, which leverages inductive learning to eliminate randomness in optimization to guarantee identifiability and captures the monotonicity between overall response data distribution and cognitive states to prevent explainability overfitting.

Abstract

Personalized learner modeling using cognitive diagnosis (CD), which aims to model learners' cognitive states by diagnosing learner traits from behavioral data, is a fundamental yet significant task in many web learning services. Existing cognitive diagnosis models (CDMs) follow the proficiency-response paradigm that views learner traits and question parameters as trainable embeddings and learns them through learner performance prediction. However, we notice that this paradigm leads to the inevitable non-identifiability and explainability overfitting problem, which is harmful to the quantification of learners' cognitive states and the quality of web learning services. To address these problems, we propose an identifiable cognitive diagnosis framework (ID-CDF) based on a novel response-proficiency-response paradigm inspired by encoder-decoder models. Specifically, we first devise the diagnostic module of ID-CDF, which leverages inductive learning to eliminate randomness in optimization to guarantee identifiability and captures the monotonicity between overall response data distribution and cognitive states to prevent explainability overfitting. Next, we propose a flexible predictive module for ID-CDF to ensure diagnosis preciseness. We further present an implementation of ID-CDF, i.e., ID-CDM, to illustrate its usability. Extensive experiments on four real-world datasets with different characteristics demonstrate that ID-CDF can effectively address the problems without loss of diagnosis preciseness.
Paper Structure (19 sections, 16 equations, 13 figures, 2 tables)

This paper contains 19 sections, 16 equations, 13 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Preliminary
  4. Mathematical Notations and Task Definition
  5. Identifiability and Explainability of CDMs
  6. The Structure of ID-CDF
  7. ID-CDM: An Implementation of ID-CDF
  8. Experiment
  9. Experiment Overview
  10. Expermental Setup
  11. Identifiability Evaluation (RQ1)
  12. Explainability Evaluation (RQ2)
  13. Learner Score Prediction (RQ3)
  14. Learner traits clustering (RQ4)
  15. Conclusion
  16. ...and 4 more sections

Figures (13)

  • Figure 1: An example of existing CD-based learner modeling. Learner traits ($\Theta$) and question parameters ($\Psi$) are fitted on the response data through transductive learning.
  • Figure 2: The histogram of the Manhattan distance of diagnostic results of learners with the same response distribution in Math1 dataset, diagnosed by NCDM WangF2022.
  • Figure 3: The classical P-R paradigm and our proposed R-P-R paradigm in personalized learner modeling using CD.
  • Figure 4: The structure of identifiable cognitive diagnosis framework (ID-CDF).
  • Figure 5: Results of the explainability of diagnosed learner traits (RQ2).
  • ...and 8 more figures

Theorems & Definitions (6)

  • definition 1
  • definition 2
  • definition 3
  • definition 4
  • definition 5
  • definition 6