Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Creating a Trajectory for Code Writing: Algorithmic Reasoning Tasks

Shruthi Ravikumar, Margaret Hamilton, Charles Thevathayan, Maria Spichkova, Kashif Ali, Gayan Wijesinghe

TL;DR

This work tackles why novices struggle with code writing by proposing Algorithmic Reasoning Tasks (ARTs) that embed varying cognitive demands to map a learning trajectory from problem understanding to coding. ARTs are designed in Detection, Comparison, and Analysis varieties and are validated against three code-writing tasks using Random Forest and Ordinal Logistic Regression models to predict code-writing performance. Results show ART-type tasks correlate more strongly with code writing than traditional tracing tasks, with RF achieving around 84.5–85.5% accuracy in prediction and ART Comparison emerging as the most predictive among ART types. The study demonstrates a scalable, automated assessment framework capable of early identification of at-risk students and guiding instructional interventions to improve coding skills.

Abstract

Many students in introductory programming courses fare poorly in the code writing tasks of the final summative assessment. Such tasks are designed to assess whether novices have developed the analytical skills to translate from the given problem domain to coding. In the past researchers have used instruments such as code-explain and found that the extent of cognitive depth reached in these tasks correlated well with code writing ability. However, the need for manual marking and personalized interviews used for identifying cognitive difficulties limited the study to a small group of stragglers. To extend this work to larger groups, we have devised several question types with varying cognitive demands collectively called Algorithmic Reasoning Tasks (ARTs), which do not require manual marking. These tasks require levels of reasoning which can define a learning trajectory. This paper describes these instruments and the machine learning models used for validating them. We have used the data collected in an introductory programming course in the penultimate week of the semester which required attempting ART type instruments and code writing. Our preliminary research suggests ART type instruments can be combined with specific machine learning models to act as an effective learning trajectory and early prediction of code-writing skills.

Creating a Trajectory for Code Writing: Algorithmic Reasoning Tasks

TL;DR

This work tackles why novices struggle with code writing by proposing Algorithmic Reasoning Tasks (ARTs) that embed varying cognitive demands to map a learning trajectory from problem understanding to coding. ARTs are designed in Detection, Comparison, and Analysis varieties and are validated against three code-writing tasks using Random Forest and Ordinal Logistic Regression models to predict code-writing performance. Results show ART-type tasks correlate more strongly with code writing than traditional tracing tasks, with RF achieving around 84.5–85.5% accuracy in prediction and ART Comparison emerging as the most predictive among ART types. The study demonstrates a scalable, automated assessment framework capable of early identification of at-risk students and guiding instructional interventions to improve coding skills.

Abstract

Many students in introductory programming courses fare poorly in the code writing tasks of the final summative assessment. Such tasks are designed to assess whether novices have developed the analytical skills to translate from the given problem domain to coding. In the past researchers have used instruments such as code-explain and found that the extent of cognitive depth reached in these tasks correlated well with code writing ability. However, the need for manual marking and personalized interviews used for identifying cognitive difficulties limited the study to a small group of stragglers. To extend this work to larger groups, we have devised several question types with varying cognitive demands collectively called Algorithmic Reasoning Tasks (ARTs), which do not require manual marking. These tasks require levels of reasoning which can define a learning trajectory. This paper describes these instruments and the machine learning models used for validating them. We have used the data collected in an introductory programming course in the penultimate week of the semester which required attempting ART type instruments and code writing. Our preliminary research suggests ART type instruments can be combined with specific machine learning models to act as an effective learning trajectory and early prediction of code-writing skills.
Paper Structure (15 sections, 6 figures, 5 tables)

This paper contains 15 sections, 6 figures, 5 tables.

Table of Contents

  1. INTRODUCTION
  2. BACKGROUND AND RELATED WORKS
  3. Approaches to assess programming skills in early stages of learning
  4. Application of machine learning for prediction of students' performance
  5. METHODOLOGY
  6. Design of an Abstract Reasoning Task (ART) Type Question
  7. Code Writing
  8. Marking
  9. Choice of Machine Learning Model
  10. Data Exploration and Training
  11. Data Preparation
  12. Data Training and Performance Metrics
  13. Performance Metrics
  14. RESULTS
  15. CONCLUSION AND FUTURE WORK

Figures (6)

  • Figure 1: ART Detection Type question used in the study
  • Figure 2: ART Comparison Type question used in the study
  • Figure 3: ART Analysis Type question used in the study
  • Figure 4: Performance metrics of RF and LF models in predicting the students scoring 0 marks
  • Figure 5: Performance metrics of RF and LF models in predicting the students scoring 0.5 marks
  • ...and 1 more figures