Toward Finding and Supporting Struggling Students in a Programming Course with an Early Warning System

TL;DR

The study investigates how cognitive skills relate to learning programming and whether an early-warning system can identify at-risk students early enough for effective intervention. It examines an array of predictors (Early-MM mental models, language skills, attention, and fluid intelligence) alongside a syntax drill-and-practice intervention in a two-semester course, using final-exam performance as the outcome. Results show only modest predictive power for most cognitive measures, with Early-MM and language skills offering the most promise, while attention shows limited utility; syntax drills significantly improve final performance for participants. The work demonstrates the feasibility of pre-course or early-course screening to tailor support, though statistical modeling was hampered by small, incomplete samples, underscoring the need for larger-scale replication and refinement of the predictive model.

Abstract

Background: Programming skills are advantageous to navigate today's society, so it is important to teach them to students. However, failure rates for programming courses are high, and especially students who fall behind early in introductory programming courses tend to stay behind. Objective: To catch these students as early as possible, we aim to develop an early warning system, so we can offer the students support, for example, in the form of syntax drill-and-practice exercises. Method: To develop the early warning system, we assess different cognitive skills of students of an introductory programming course. On several points in time over the course, students complete tests that measure their ability to develop a mental model of programming, language skills, attention, and fluid intelligence. Then, we evaluated to what extent these skills predict whether students acquire programming skills. Additionally, we assess how syntax drill-and-practice exercises improve how students acquire programming skill. Findings: Most of the cognitive skills can predict whether students acquire programming skills to a certain degree. Especially the ability to develop an early mental model of programming and language skills appear to be relevant. Fluid intelligence also shows predictive power, but appears to be comparable with the ability to develop a mental model. Furthermore, we found a significant positive effect of the syntax drill-and-practice exercises on the success of a course. Implications: Our first suggestion of an early warning system consists of few, easy-to-apply tests that can be integrated in programming courses or applied even before a course starts. Thus, with the start of a programming course, students who are at high risk of failing can be identified and offered support, for example, in the form of syntax drill-and-practice exercises to help students to develop programming skills.

Figures (8)

• Figure 1: Tasks of the Early-MM ahadi_falling_2014.
• Figure 2: One item in the C-Test.
• Figure 3: Possible items of the R-Voc (a) and the P-Voc (b).
• Figure 4: The task is to react as quickly as possible to any d with two lines. The target symbols are highlighted with thicker boxes for purpose of illustration.
• Figure 5: DESIGMA-A: Participants need to compose the missing figure of the matrix. In this example, the correct solution is a circle, which can be composed by the elements of the second column. becker2014matrizenkonstruktionsaufgabe