Pattern-based Knowledge Component Extraction from Student Code Using Representation Learning

TL;DR

The paper tackles the problem of explainable, fine-grained knowledge modeling in CS education by extracting pattern-based KCs directly from student code. It introduces a pipeline that combines a Subtree-based Attention Neural Network (SANN) to identify important AST substructures with a beta-Variational Autoencoder (VAE) to learn latent representations, followed by K-means clustering to form 50 pattern-based KCs. These KCs are validated through learning-curve analyses and integration into Deep Knowledge Tracing (KC-DKT), achieving meaningful learning trajectories and a substantial improvement in predictive accuracy (KC-DKT: $AUC = 74.77\%$ vs DKT: $AUC = 65.87\%$). The approach yields explainable, scalable knowledge modeling from programs, enabling targeted instruction and adaptive feedback in large-scale CS education settings.

Abstract

Effective personalized learning in computer science education depends on accurately modeling what students know and what they need to learn. While Knowledge Components (KCs) provide a foundation for such modeling, automated KC extraction from student code is inherently challenging due to insufficient explainability of discovered KCs and the open-endedness of programming problems with significant structural variability across student solutions and complex interactions among programming concepts. In this work, we propose a novel, explainable framework for automated KC discovery through pattern-based KCs: recurring structural patterns within student code that capture the specific programming patterns and language constructs that students must master. Toward this, we train a Variational Autoencoder to generate important representative patterns from student code guided by an explainable, attention-based code representation model that identifies important correct and incorrect pattern implementations from student code. These patterns are then clustered to form pattern-based KCs. We evaluate our KCs using two well-established methods informed by Cognitive Science: learning curve analysis and Deep Knowledge Tracing (DKT). Experimental results demonstrate meaningful learning trajectories and significant improvements in DKT predictive performance over traditional KT methods. This work advances knowledge modeling in CS education by providing an automated, scalable, and explainable framework for identifying granular code patterns and algorithmic constructs, essential for student learning.

Figures (6)

• Figure 1: Model architecture of SANN.
• Figure 2: Model architecture of the VAE model.
• Figure 3: KC-DKT model with pattern-based KC vectors.
• Figure 4: Learning curves with good learning trend. Node colors indicate opportunity coverage: the percentage of students who encountered the KC at a given opportunity.
• Figure 5: Learning curve with high error/still learning trend.