Rubric-based Learner Modelling via Noisy Gates Bayesian Networks for Computational Thinking Skills Assessment

TL;DR

The paper tackles rubric-based learner assessment by integrating rubrics into Bayesian networks with two key innovations: enforcing the rubric's competency ordering via dummy constraint nodes and modeling supplementary skills through a two-layer gate structure (disjunctive noisy-OR followed by a conjunctive AND). This yields a compact, interpretable framework that can produce real-time inferences while maintaining alignment with expert rubric definitions. Applied to the Cross Array Task dataset (CAT) with 109 pupils, four model variants (B, BC, BCS, ECS) show high correspondence with expert scores and offer richer posterior information about individual skills and supplementary competencies. The approach supports scalable, rubric-driven ITS/ITAS implementations by reducing elicitation burden and improving diagnostic fidelity, with ongoing work on learning parameters from interaction data.

Abstract

In modern and personalised education, there is a growing interest in developing learners' competencies and accurately assessing them. In a previous work, we proposed a procedure for deriving a learner model for automatic skill assessment from a task-specific competence rubric, thus simplifying the implementation of automated assessment tools. The previous approach, however, suffered two main limitations: (i) the ordering between competencies defined by the assessment rubric was only indirectly modelled; (ii) supplementary skills, not under assessment but necessary for accomplishing the task, were not included in the model. In this work, we address issue (i) by introducing dummy observed nodes, strictly enforcing the skills ordering without changing the network's structure. In contrast, for point (ii), we design a network with two layers of gates, one performing disjunctive operations by noisy-OR gates and the other conjunctive operations through logical ANDs. Such changes improve the model outcomes' coherence and the modelling tool's flexibility without compromising the model's compact parametrisation, interpretability and simple experts' elicitation. We used this approach to develop a learner model for Computational Thinking (CT) skills assessment. The CT-cube skills assessment framework and the Cross Array Task (CAT) are used to exemplify it and demonstrate its feasibility.

Figures (7)

• Figure 1: Example of BN-based learner model.
• Figure 2: A noisy gate explicit formulation (adapted form Anonymous2022).
• Figure 3: Example of a BN network modelling a task-specific assessment rubric with two cells, represented by skills $X_1$ and $X_2$ (on the right), $m$ supplementary skills grouped in a single set (on the left), and the constraint $X_2\implies X_1$, represented by the auxiliary variable $D_1$ (on the top right).
• Figure 4: CAT experimental settings (adapted from piatti_2022).
• Figure 5: The 12 CAT schemes $T$ (top); the values of the inhibition parameters $\lambda^t_{rc}$ for the target skill nodes (centre); the value of the inhibition parameters $\lambda^t_{S_i}$ for the supplementary skill nodes (bottom). The inhibition parameters for both the target and supplementary skill are depicted as a matrix of nine rows representing the answers, and as many columns as the number of modelled skills. The strength of the skill-answer relation has eleven levels, from $0.1$ to $0.6$ with a step of $0.05$. Darker shades of grey mean lower skill-answer inhibition probabilities and white squares denote non-relevant skills.