An Online Integrated Development Environment for Automated Programming Assessment Systems
Eduard Frankford, Daniel Crazzolara, Michael Vierhauser, Niklas Meissner, Stephan Krusche, Ruth Breu
TL;DR
The paper tackles the lack of feature-rich, browser-based IDEs within Automated Programming Assessment Systems (APASs) and proposes a prototypical open-source online IDE integrated with the Artemis platform. It derives requirements from desktop IDE analysis and a survey of APAS users, then implements a scalable architecture using a shared Language Server Protocol (LSP) server, load balancing, a central VCS, and Docker sandboxing to enable secure code execution. The evaluation, based on Technology Acceptance Model (TAM) and memory measurements with 27 first-year students, shows the solution is easy to learn but yields mixed perceptions of usefulness, while demonstrating memory scalability and signaling efficiency improvements via debouncing. The work contributes practical architectural strategies for memory-efficient, secure, and scalable online IDEs in educational settings and motivates future enhancements such as integrated debuggers and advanced code assistance to broaden applicability beyond introductory tasks.
Abstract
The increasing demand for programmers has led to a surge in participants in programming courses, making it increasingly challenging for instructors to assess student code manually. As a result, automated programming assessment systems (APASs) have been developed to streamline this process. These APASs support lecturers by managing and evaluating student programming exercises at scale. However, these tools often do not provide feature-rich online editors compared to their traditional integrated development environments (IDEs) counterparts. This absence of key features, such as syntax highlighting and autocompletion, can negatively impact the learning experience, as these tools are crucial for effective coding practice. To address this gap, this research contributes to the field of programming education by extracting and defining requirements for an online IDE in an educational context and presenting a prototypical implementation of an open-source solution for a scalable and secure online IDE. The usability of the new online IDE was assessed using the Technology Acceptance Model (TAM), gathering feedback from 27 first-year students through a structured survey. In addition to these qualitative insights, quantitative measures such as memory (RAM) usage were evaluated to determine the efficiency and scalability of the tool under varying usage conditions.