Requirements-Based Test Generation: A Comprehensive Survey

TL;DR

This survey comprehensively maps 267 studies on Requirements-Based Test Generation (RBTG) up to 2024, covering input requirement types, generation approaches (direct versus via intermediate representations), test-case forms (ATCs, CTCs, TSs), supporting tools, evaluation methods, and practical domains. It reveals a historical shift from model-based, formal/semi-formal methods toward NL-driven approaches, while highlighting a persistent gap in translating abstract test cases into executable ones and in rigorous, real-world evaluation. The work identifies key challenges—quality of requirements, executable test generation, benchmarks, evaluation rigor, and industrial adoption—and proposes future directions, including LLM-driven RBTG and non-functional RBTG integration. Overall, the paper provides a foundational, evidence-based synthesis to guide researchers and practitioners toward more automated, scalable, and industry-relevant RBTG methods. The findings underscore the potential that advances in NLP/LLMs, modeling techniques, and standardized benchmarks hold for making RBTG more practical and impactful.

Abstract

As an important way of assuring software quality, software testing generates and executes test cases to identify software failures. Many strategies have been proposed to guide test-case generation, such as source-code-based approaches and methods based on bug reports. Requirements-based test generation (RBTG) constructs test cases based on specified requirements, aligning with user needs and expectations, without requiring access to the source code. Since its introduction in 1994, there have been many contributions to the development of RBTG, including various approaches, implementations, tools, assessment and evaluation methods, and applications. This paper provides a comprehensive survey on RBTG, categorizing requirement types, classifying approaches, investigating types of test cases, summarizing available tools, and analyzing experimental evaluations. This paper also summarizes the domains and industrial applications of RBTG, and discusses some open research challenges and potential future work.

Figures (12)

• Figure 1: Requirements-based test-generation process.
• Figure 2: Overview of the surveyed 267 RBTG studies' publication.
• Figure 3: Venue distribution for RBTG studies.
• Figure 4: Types of requirements.
• Figure 5: Distribution of types of natural language requirements.