On the Impact of 3D Visualization of Repository Metrics in Software Engineering Education

TL;DR

This registered report tackles the challenge of teaching software process comprehension by evaluating whether VR visualization of repository metrics enhances SE education relative to traditional 2D visualization. The study proposes a rigorous within-subject AB/BA crossover experiment using two data sources (CHAOSS and OpenShift) and two visualization modalities (2D Kibana and VR BabiaXR), measuring learning outcomes, task accuracy/time, and the quality of the learning experience with validated instruments. The analysis plan combines mixed linear models with normality checks and potential GLMMs, plus construct validation via Exploratory Factor Analysis, and includes careful consideration of ethical and validity threats. If VR proves advantageous, the work could influence curriculum design and the adoption of immersive data visualization tools to prepare students for industry tasks in software maintenance, code review, and CI/CD processes.

Abstract

Context: Software development is a complex socio-technical process requiring a deep understanding of various aspects. In order to support practitioners in understanding such a complex activity, repository process metrics, like number of pull requests and issues, emerged as crucial for evaluating CI/CD workflows and guiding informed decision-making. The research community proposed different ways to visualize these metrics to increase their impact on developers' process comprehension: VR is a promising one. Nevertheless, despite such promising results, the role of VR, especially in educational settings, has received limited research attention. Objective: This study aims to address this gap by exploring how VR-based repository metrics visualization can support the teaching of process comprehension. Method: The registered report proposes the execution of a controlled experiment where VR and non-VR approaches will be compared, with the final aim to assess whether repository metrics in VR's impact on learning experience and software process comprehension. By immersing students in an intuitive environment, this research hypothesizes that VR can foster essential analytical skills, thus preparing software engineering students more effectively for industry requirements and equipping them to navigate complex software development tasks with enhanced comprehension and critical thinking abilities.