Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating Micro Parsons Problems as Exam Questions

Zihan Wu, David H. Smith

TL;DR

This study evaluates micro Parsons problems as exam items in an introductory Python course, comparing their psychometric properties to single-line code writing and examining student perceptions. Using Item Response Theory, specifically a 2PL framework, the authors estimate item difficulty and discrimination across four exams and two problem formats, with a supplementary survey of 390 students to capture experiences and preferences. They find micro Parsons are generally low in difficulty and moderately discriminating, similar to related single-line items, but with notable exceptions (e.g., a file-open task) and a tendency for distractor design to influence item properties. The work provides actionable guidance for exam design, highlighting the value of partial credit and instant feedback while cautioning about distractor quality and potential expertise reversal effects when applying micro Parsons in summative assessments.

Abstract

Parsons problems are a type of programming activity that present learners with blocks of existing code and requiring them to arrange those blocks to form a program rather than write the code from scratch. Micro Parsons problems extend this concept by having students assemble segments of code to form a single line of code rather than an entire program. Recent investigations into micro Parsons problems have primarily focused on supporting learners leaving open the question of micro Parsons efficacy as an exam item and how students perceive it when preparing for exams. To fill this gap, we included a variety of micro Parsons problems on four exams in an introductory programming course taught in Python. We use Item Response Theory to investigate the difficulty of the micro Parsons problems as well as the ability of the questions to differentiate between high and low ability students. We then compare these results to results for related questions where students are asked to write a single line of code from scratch. Finally, we conduct a thematic analysis of the survey responses to investigate how students' perceptions of micro Parsons both when practicing for exams and as they appear on exams.

Evaluating Micro Parsons Problems as Exam Questions

TL;DR

This study evaluates micro Parsons problems as exam items in an introductory Python course, comparing their psychometric properties to single-line code writing and examining student perceptions. Using Item Response Theory, specifically a 2PL framework, the authors estimate item difficulty and discrimination across four exams and two problem formats, with a supplementary survey of 390 students to capture experiences and preferences. They find micro Parsons are generally low in difficulty and moderately discriminating, similar to related single-line items, but with notable exceptions (e.g., a file-open task) and a tendency for distractor design to influence item properties. The work provides actionable guidance for exam design, highlighting the value of partial credit and instant feedback while cautioning about distractor quality and potential expertise reversal effects when applying micro Parsons in summative assessments.

Abstract

Parsons problems are a type of programming activity that present learners with blocks of existing code and requiring them to arrange those blocks to form a program rather than write the code from scratch. Micro Parsons problems extend this concept by having students assemble segments of code to form a single line of code rather than an entire program. Recent investigations into micro Parsons problems have primarily focused on supporting learners leaving open the question of micro Parsons efficacy as an exam item and how students perceive it when preparing for exams. To fill this gap, we included a variety of micro Parsons problems on four exams in an introductory programming course taught in Python. We use Item Response Theory to investigate the difficulty of the micro Parsons problems as well as the ability of the questions to differentiate between high and low ability students. We then compare these results to results for related questions where students are asked to write a single line of code from scratch. Finally, we conduct a thematic analysis of the survey responses to investigate how students' perceptions of micro Parsons both when practicing for exams and as they appear on exams.
Paper Structure (23 sections, 1 equation, 5 figures)

This paper contains 23 sections, 1 equation, 5 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Parsons and Micro Parsons Problems
  4. Parsons Problems as Exam Items
  5. Measurement Theory
  6. Methods
  7. Course Context and Data Collection
  8. Problem Design
  9. Student Survey
  10. RQ1: Difficulty and Discrimination
  11. Results
  12. RQ2: Student Perceptions and Preferences
  13. Results
  14. Perceived Difficulty
  15. Assisting Learning
  16. ...and 8 more sections

Figures (5)

  • Figure 1: The two variations of Parsons problems used in the course.
  • Figure 2: An example of a single-line programming question
  • Figure 3: The Item-Difficulty and Item-Discrimination statistics for the items used in this study.
  • Figure 4: Comparison of the distributions of item-difficulty and item-discrimination statistics for micro Parsons and single-line code writing questions
  • Figure 5: The feedback and partial credit mechanism used for our implementation of micro Parsons.