A Tool for Test Case Scenarios Generation Using Large Language Models

TL;DR

The paper addresses the automation of test case scenario generation from software requirements by introducing a web-based tool that uses an LLM-based agent and prompt engineering to derive test scenarios from epics and user stories. Built on a React/Flask/OpenAI stack, the tool generates scenarios in JSON, exports CSV for integration with testing tools, and demonstrates a rapid ~2-second generation per requirement. The work extends an existing tool with GPT-3.5 to translate prioritized requirements into test artifacts, while evaluating outputs for conformance and documenting initial findings. It also discusses challenges such as hallucinations and cost, and outlines future directions including benchmarking open-source LLMs, code-generation for test cases, and co-pilot features to streamline testing workflows.

Abstract

Large Language Models (LLMs) are widely used in Software Engineering (SE) for various tasks, including generating code, designing and documenting software, adding code comments, reviewing code, and writing test scripts. However, creating test scripts or automating test cases demands test suite documentation that comprehensively covers functional requirements. Such documentation must enable thorough testing within a constrained scope and timeframe, particularly as requirements and user demands evolve. This article centers on generating user requirements as epics and high-level user stories and crafting test case scenarios based on these stories. It introduces a web-based software tool that employs an LLM-based agent and prompt engineering to automate the generation of test case scenarios against user requirements.

Figures (2)

• Figure 1: Process of Generating testCase Scenarios Using LLMs and Analyzing Its Output
• Figure 2: Test Scenarios generation Using LLMs