A Block-Based Testing Framework for Scratch

TL;DR

This paper presents a Scratch extension that integrates automated, block-based testing directly into the Scratch environment. By adding four categories of test blocks—control, trigger, assertion, and reporter—the framework enables writing, executing, and validating tests without external tooling, including a test-control interface and batch testing capabilities. An empirical study with 28 teachers demonstrates high effectiveness (median test-accuracy ≈ 0.93 and assessment-accuracy ≈ 0.92) and positive reception, supporting automated feedback and streamlined evaluation of student Scratch projects. The work offers practical impact for educators and a foundation for future enhancements in test-driven learning and scalable feedback within visual programming environments.

Abstract

Block-based programming environments like Scratch are widely used in introductory programming courses. They facilitate learning pivotal programming concepts by eliminating syntactical errors, but logical errors that break the desired program behaviour are nevertheless possible. Finding such errors requires testing, i.e., running the program and checking its behaviour. In many programming environments, this step can be automated by providing executable tests as code; in Scratch, testing can only be done manually by invoking events through user input and observing the rendered stage. While this is arguably sufficient for learners, the lack of automated testing may be inhibitive for teachers wishing to provide feedback on their students' solutions. In order to address this issue, we introduce a new category of blocks in Scratch that enables the creation of automated tests. With these blocks, students and teachers alike can create tests and receive feedback directly within the Scratch environment using familiar block-based programming logic. To facilitate the creation and to enable batch processing sets of student solutions, we extend the Scratch user interface with an accompanying test interface. We evaluated this testing framework with 28 teachers who created tests for a popular Scratch game and subsequently used these tests to assess and provide feedback on student implementations. An overall accuracy of 0.93 of teachers' tests compared to manually evaluating the functionality of 21 student solutions demonstrates that teachers are able to create and effectively use tests. A subsequent survey confirms that teachers consider the block-based test approach useful.

Figures (11)

• Figure 1: A correct and a faulty variant of an initialisation script are tested.
• Figure 2: New blocks introduced for testing.
• Figure 3: The passing test D uses a Yield block to give execution precedence to script A.
• Figure 4: The passing test waits until the script containing the movement logic has fully completed.
• Figure 5: The examples window showcases tests for common scenarios and allows users to import them into their project.