Quality in model-driven engineering: a tertiary study
Miguel Goulão, Vasco Amaral, Marjan Mernik
TL;DR
This paper addresses the fragmented evidence on how Model-Driven Engineering (MDE) impacts software quality by conducting a tertiary study that maps 22 systematic reviews and mapping studies. It catalogues the scope, quality assessment approaches, objects of study, and ISO 25010 attributes addressed, revealing a predominance of maintainability and a dearth of evidence on quality in use. The findings show that most questions in the secondary studies are mapping-oriented rather than direct comparisons, underscoring the need for more empirical, comparative research and practitioner-focused guidance. The authors propose a research roadmap to advance industry-relevant, evidence-based understanding of MDE's impact on quality and to close gaps in transformations and process-related quality attributes.
Abstract
Model-driven engineering (MDE) is believed to have a significant impact in software quality. However, researchers and practitioners may have a hard time locating consolidated evidence on this impact, as the available information is scattered in several different publications. Our goal is to aggregate consolidated findings on quality in MDE, facilitating the work of researchers and practitioners in learning about the coverage and main findings of existing work as well as identifying relatively unexplored niches of research that need further attention. We performed a tertiary study on quality in MDE, in order to gain a better understanding of its most prominent findings and existing challenges, as reported in the literature. We identified 22 systematic literature reviews and mapping studies and the most relevant quality attributes addressed by each of those studies, in the context of MDE. Maintainability is clearly the most often studied and reported quality attribute impacted by MDE. Eighty out of 83 research questions in the selected secondary studies have a structure that is more often associated with mapping existing research than with answering more concrete research questions (e.g., comparing two alternative MDE approaches with respect to their impact on a specific quality attribute). We briefly outline the main contributions of each of the selected literature reviews. In the collected studies, we observed a broad coverage of software product quality, although frequently accompanied by notes on how much more empirical research is needed to further validate existing claims. Relatively, little attention seems to be devoted to the impact of MDE on the quality in use of products developed using MDE.