MindScratch: A Visual Programming Support Tool for Classroom Learning Based on Multimodal Generative AI

TL;DR

MindScratch proposes a multimodal generative AI-driven visual programming tool that uses interactive mind maps to align student projects with teacher-defined learning objectives while providing staged guidance and multimodal assets. In a within-subject study with 24 fifth graders, MindScratch outperformed Scratch in achieving objectives, improving computational thinking and creativity, and generating richer mind maps; educators highlighted benefits and addressed safety and workflow concerns. The approach combines stage-based dialogue, relationship annotation, and scaffolded code assistance with a safety-focused generation pipeline, showing strong potential to reduce teacher workload while enhancing learning outcomes. The work offers design considerations and a roadmap for integrating AI-driven resources into classroom programming, with implications for scalability, reliability, and ethical use in AI-supported curricula.

Abstract

Programming has become an essential component of K-12 education and serves as a pathway for developing computational thinking skills. Given the complexity of programming and the advanced skills it requires, previous research has introduced user-friendly tools to support young learners. However, our interviews with six programming educators revealed that current tools often fail to reflect classroom learning objectives, offer flexible, high-quality guidance, and foster student creativity. This highlights the need for more adaptive and reflective tools. Therefore, we introduced MindScratch, a multimodal generative AI (GAI) powered visual programming support tool. MindScratch aims to balance structured classroom activities with free programming creation, supporting students in completing creative programming projects based on teacher-set learning objectives while also providing programming scaffolding. Our user study results indicate that, compared to the baseline, MindScratch more effectively helps students achieve high-quality projects aligned with learning objectives. It also enhances students' computational thinking skills and creative thinking. Overall, we believe that GAI-driven educational tools like MindScratch offer students a focused and engaging learning experience.

Figures (7)

• Figure 1: MindScratch's User Interface. In the mind map (a) with block palette (a4), each node represents the collaborative creation of students, teachers, and AI (a1, a2, a3). The connections between nodes are annotated, and the highlighted nodes indicate their relevance to learning objectives (a5, a6). Within the dialogue box (b), the system provides structured guidance and real-time support through a Q&A mechanism (b1, b2). A floating text input box allows for the generation of audio materials through text editing (a7).
• Figure 2: MindScratch's drawing board. It enables students to create, refine, and save image materials (c1, c2, c3).
• Figure 3: MindScratch user interaction process: Teachers set learning objectives and programming themes to prompt the LLM and guide student creation. Students collaborate with MindScratch to create characters, generate images and audio, and add them to the mind map. In the coding phase, students use LLM-provided logic and code blocks for implementation. Finally, students conduct hands-on Scratch programming and can ask MindScratch for help if needed.
• Figure 4: Overview of the logic-code assistance pipeline for MindScratch.
• Figure 5: The score distribution between MindScratch and Scratch. Note that higher values indicate positive feedback, and vice versa.