Automated Test Production -- Systematic Literature Review

TL;DR

This SLR surveys Automated Test Production (ATP) for computer programs, mapping models, methodologies, and tools while assessing their applicability. It follows established SLR practices (e.g., Brereton, Basili, Keele, Chen, Keele) to define research questions and a qualitative analysis protocol. Findings reveal a strong emphasis on white-box verification and data generation, persistent Oracle challenges, and a fragmented tool landscape that suggests limited maturity. The work catalogs concrete contributions (methods, tools, metrics) and outlines future directions to strengthen ATP practice and validation across broader contexts.

Abstract

Identifying the main contributions related to the Automated Test Production (ATP) of Computer Programs and providing an overview about models, methodologies and tools used for this purpose is the aim of this Systematic Literature Review (SLR). The results will enable a comprehensive analysis and insight to evaluate their applicability. A previously produced Systematic Literature Mapping (SLM) contributed to the formulation of the ``Research Questions'' and parameters for the definition of the qualitative analysis protocol of this review.

Figures (9)

• Figure 1: Steps to perform an SLR (adaptedo from brereton2007lessons)
• Figure 2: Qualified studies
• Figure 3: Classification Scheme (adapted from petersen2008systematic)
• Figure 4: Artifact Types
• Figure 5: Generators Types