Enhancing Software Testing Education: Understanding Where Students Struggle

TL;DR

The paper investigates which software testing concepts students struggle with and how these misconceptions drive ineffective test-suite revisions despite automated concept-level feedback. It analyzes two senior-level assignments in a QA course using a web-based feedback tool that maps missing tests to fundamental concepts, comparing conceptual feedback to later detailed feedback when progress stalls. Results show that decision coverage and branch coverage are the most persistent challenges, with exception handling also rising in difficulty; equivalence partitioning becomes a major concern in the second assignment. The findings highlight a tendency toward superficial test changes and poor coverage strategies, underscoring the need for structured, concept-focused instruction, adaptive feedback, and metacognitive practices to improve the design of robust test suites in software engineering education.

Abstract

Effective software testing is critical for producing reliable and secure software, yet many computer science students struggle to master the foundational concepts required to construct comprehensive test suites. While automated feedback tools are widely used to support student learning, it remains unclear which testing concepts are most frequently misunderstood and how these misunderstandings are reflected in students' test suite revisions. This study examines the specific testing concepts that lead students to make ineffective changes, those that fail to improve code coverage, during test suite development. Leveraging an automated feedback tool in a senior-level software testing course, we analyzed student submissions from two assignments to identify prevalent conceptual gaps and patterns of unproductive modification. Our results reveal that decision coverage and exception handling are persistent challenges, and that students most often make superficial or method-level changes that do not enhance coverage. These findings provide actionable insights for educators, researchers, and tool designers. By pinpointing the concepts that most often contribute to poor testing outcomes, we can refine feedback systems, target instruction to address persistent misconceptions, and more effectively support students in developing robust, maintainable test suites.