Exploring Heterogeneity and Uncertainty for Graph-based Cognitive Diagnosis Models in Intelligent Education

TL;DR

This work tackles the limitations of graph-based cognitive diagnosis models caused by edge heterogeneity and uncertain response semantics. It introduces ISG-CD, which integrates a semantic-aware GNN to leverage heterogeneous edge types and an informative edge differentiation layer grounded in the information bottleneck to produce a reliable graph $ ilde{\mathcal{G}}$. The training objective combines a Binary Cross-Entropy loss (stemming from a lower bound on mutual information $I({R};\mathbf{U},\mathbf{V},\tilde{\mathcal{G}})$) with a Hilbert–Schmidt Independence Criterion term to minimize $I(\tilde{\mathcal{G}},\mathcal{G})$, enabling unsupervised edge pruning. Empirical results on three real-world datasets show ISG-CD consistently outperforms strong baselines in ACC, AUC, and DOA, validating the efficacy of modeling edge semantics and mitigating uncertain edges for reliable cognitive diagnosis.

Abstract

Graph-based Cognitive Diagnosis (CD) has attracted much research interest due to its strong ability on inferring students' proficiency levels on knowledge concepts. While graph-based CD models have demonstrated remarkable performance, we contend that they still cannot achieve optimal performance due to the neglect of edge heterogeneity and uncertainty. Edges involve both correct and incorrect response logs, indicating heterogeneity. Meanwhile, a response log can have uncertain semantic meanings, e.g., a correct log can indicate true mastery or fortunate guessing, and a wrong log can indicate a lack of understanding or a careless mistake. In this paper, we propose an Informative Semantic-aware Graph-based Cognitive Diagnosis model (ISG-CD), which focuses on how to utilize the heterogeneous graph in CD and minimize effects of uncertain edges. Specifically, to explore heterogeneity, we propose a semantic-aware graph neural networks based CD model. To minimize effects of edge uncertainty, we propose an Informative Edge Differentiation layer from an information bottleneck perspective, which suggests keeping a minimal yet sufficient reliable graph for CD in an unsupervised way. We formulate this process as maximizing mutual information between the reliable graph and response logs, while minimizing mutual information between the reliable graph and the original graph. After that, we prove that mutual information maximization can be theoretically converted to the classic binary cross entropy loss function, while minimizing mutual information can be realized by the Hilbert-Schmidt Independence Criterion. Finally, we adopt an alternating training strategy for optimizing learnable parameters of both the semantic-aware graph neural networks based CD model and the edge differentiation layer. Extensive experiments on three real-world datasets have demonstrated the effectiveness of ISG-CD.

Figures (6)

• Figure 1: (a) The process of educational CD. These CD models take students' responses logs on exercises, and exercise-concept relations as input, and output students' proficiency levels on all knowledge concepts. (b) GNN between two students (John and Mark). Their interactions with exercises have extensively overlapped, however, they are not similar from a graph view due to heterogeneity. Further, there are uncertain edges, e.g., according to interactions between John and exercises $e_3,e_4$, John has already grasped concept $k2$. John's performance on exercise $e_5$ cannot reflect his ability on concept $k2$.
• Figure 2: Semantic-aware GNN (S-GNN) based CD model.
• Figure 3: Informative Edge Differentiation (IE-Diff) Layer.
• Figure 4: Edge detection with varying the number of uncertain edges on the ASSIST and Junyi datasets.
• Figure 5: Diagnostic performance with varying number of GNN layers $L$ and temperature $t$ on the ASSIST dataset.