Supporting software engineering tasks with agentic AI: Demonstration on document retrieval and test scenario generation

TL;DR

The paper addresses automating software engineering workflows by leveraging agentic AI, focusing on two concrete tasks: automatic test-scenario generation from natural-language requirements and SE document retrieval. It proposes two architectures—a star-topology, multi-agent system for test generation and a modular, four-use-case agent framework for document processing and retrieval—implemented with LangChain/LangGraph and deployable on both on-premise and cloud LLMs. A key contribution is an end-to-end, real-world demonstration that includes a fact-checking and translation pipeline to mitigate hallucinations and ensure traceability to source content. The work has practical impact by accelerating SDLC activities and lays out a path for data-driven refinement and broader industrial adoption in intelligent software engineering.

Abstract

The introduction of large language models ignited great retooling and rethinking of the software development models. The ensuing response of software engineering research yielded a massive body of tools and approaches. In this paper, we join the hassle by introducing agentic AI solutions for two tasks. First, we developed a solution for automatic test scenario generation from a detailed requirements description. This approach relies on specialized worker agents forming a star topology with the supervisor agent in the middle. We demonstrate its capabilities on a real-world example. Second, we developed an agentic AI solution for the document retrieval task in the context of software engineering documents. Our solution enables performing various use cases on a body of documents related to the development of a single software, including search, question answering, tracking changes, and large document summarization. In this case, each use case is handled by a dedicated LLM-based agent, which performs all subtasks related to the corresponding use case. We conclude by hinting at the future perspectives of our line of research.

Figures (6)

• Figure 1: Architecture of our agentic AI solution for the test generation task, forming a star topology. The intended worker invocation order as well as possible feedback loops are also indicated, including the input and output flows. The 'robot' icon indicates agent components.
• Figure 2: An input user prompt example for our agentic AI solution for the test generation task.
• Figure 3: A chapter from a real-world FSD, which was retrieved by the retriever agent. The original image has been manually edited by translating the texts to English for reader convenience.
• Figure 4: Textual response of our test scenario generation solution for the user.
• Figure 5: A test scenario generation task output Excel file corresponding to the input prompt from Fig. \ref{['fig_prompt1']} and the input FSD from Fig. \ref{['fig_scengen_input_chapter']}.