Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Agentic LLMs for REST API Test Amplification: A Comparative Study Across Cloud Applications

Jarne Besjes, Robbe Nooyens, Tolgahan Bardakci, Mutlu Beyazit, Serge Demeyer

TL;DR

This work extends LLM-driven REST API test amplification to five cloud applications, evaluating both single-agent and multi-agent configurations to assess generalization, coverage, and efficiency. The authors combine a LangGraph-based workflow with OpenAPI retrieval and specialized planning/generation agents to produce executable, readable tests that expand endpoint, parameter, and input-space coverage while preserving semantics. Across varied architectures (production-grade and controlled environments), agentic LLM systems achieve high structural API coverage, with multi-agent setups offering broader input exploration at the cost of higher time, token, and energy expenditure. The study provides practical insights into trade-offs between accuracy, scalability, and sustainability, and outlines a path toward autonomous, CI-friendly REST API testing in complex cloud ecosystems.

Abstract

Representational State Transfer (REST) APIs are a cornerstone of modern cloud native systems. Ensuring their reliability demands automated test suites that exercise diverse and boundary level behaviors. Nevertheless, designing such test cases remains a challenging and resource intensive endeavor. This study extends prior work on Large Language Model (LLM) based test amplification by evaluating single agent and multi agent configurations across four additional cloud applications. The amplified test suites maintain semantic validity with minimal human intervention. The results demonstrate that agentic LLM systems can effectively generalize across heterogeneous API architectures, increasing endpoint and parameter coverage while revealing defects. Moreover, a detailed analysis of computational cost, runtime, and energy consumption highlights trade-offs between accuracy, scalability, and efficiency. These findings underscore the potential of LLM driven test amplification to advance the automation and sustainability of REST API testing in complex cloud environments.

Agentic LLMs for REST API Test Amplification: A Comparative Study Across Cloud Applications

TL;DR

This work extends LLM-driven REST API test amplification to five cloud applications, evaluating both single-agent and multi-agent configurations to assess generalization, coverage, and efficiency. The authors combine a LangGraph-based workflow with OpenAPI retrieval and specialized planning/generation agents to produce executable, readable tests that expand endpoint, parameter, and input-space coverage while preserving semantics. Across varied architectures (production-grade and controlled environments), agentic LLM systems achieve high structural API coverage, with multi-agent setups offering broader input exploration at the cost of higher time, token, and energy expenditure. The study provides practical insights into trade-offs between accuracy, scalability, and sustainability, and outlines a path toward autonomous, CI-friendly REST API testing in complex cloud ecosystems.

Abstract

Representational State Transfer (REST) APIs are a cornerstone of modern cloud native systems. Ensuring their reliability demands automated test suites that exercise diverse and boundary level behaviors. Nevertheless, designing such test cases remains a challenging and resource intensive endeavor. This study extends prior work on Large Language Model (LLM) based test amplification by evaluating single agent and multi agent configurations across four additional cloud applications. The amplified test suites maintain semantic validity with minimal human intervention. The results demonstrate that agentic LLM systems can effectively generalize across heterogeneous API architectures, increasing endpoint and parameter coverage while revealing defects. Moreover, a detailed analysis of computational cost, runtime, and energy consumption highlights trade-offs between accuracy, scalability, and efficiency. These findings underscore the potential of LLM driven test amplification to advance the automation and sustainability of REST API testing in complex cloud environments.

Paper Structure

This paper contains 48 sections, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. REST APIs
  5. Test amplification
  6. Large Language Models (LLMs)
  7. Generative Pre-trained Transformer (GPT)
  8. Agentic LLM systems
  9. Approaches
  10. Single-Agent Configuration
  11. Multi-Agent Configuration
  12. The planning phase
  13. The generation phase
  14. The workflow
  15. Evaluation
  16. ...and 33 more sections

Figures (2)

  • Figure 1: Single-Agent Configuration
  • Figure 2: Multi-Agent Configuration