Automated Test-Case Generation for REST APIs Using Model Inference Search Heuristic

TL;DR

The paper addresses the challenge of system-level testing for microservices, where unit-level search heuristics fail to capture inter-service interactions. It introduces MISH, a grey-box approach that continuously learns a state machine from runtime logs and uses the learned automaton to define test-case fitness, guiding EvoMaster’s evolutionary search. Through experiments on six EMB REST-enabled apps, MISH generally matches MOSA and, in several cases, surpasses it in early coverage and fault detection, despite relying on a single objective. The work demonstrates the feasibility of log-driven automaton learning for system-level REST API testing and suggests hybrid strategies to combine the strengths of single- and many-objective EAs for improved long-term exploration.

Abstract

The rising popularity of the microservice architectural style has led to a growing demand for automated testing approaches tailored to these systems. EvoMaster is a state-of-the-art tool that uses Evolutionary Algorithms (EAs) to automatically generate test cases for microservices' REST APIs. One limitation of these EAs is the use of unit-level search heuristics, such as branch distances, which focus on fine-grained code coverage and may not effectively capture the complex, interconnected behaviors characteristic of system-level testing. To address this limitation, we propose a new search heuristic (MISH) that uses real-time automaton learning to guide the test case generation process. We capture the sequential call patterns exhibited by a test case by learning an automaton from the stream of log events outputted by different microservices within the same system. Therefore, MISH learns a representation of the systemwide behavior, allowing us to define the fitness of a test case based on the path it traverses within the inferred automaton. We empirically evaluate MISH's effectiveness on six real-world benchmark microservice applications and compare it against a state-of-the-art technique, MOSA, for testing REST APIs. Our evaluation shows promising results for using MISH to guide the automated test case generation within EvoMaster.

Figures (9)

• Figure 1: Example state machine learned for a SUT $S$ used in the experiments. Running the trace $t = [10,7,5,10]$, $S$ will traverse states 0, 11, 12, 13, and end in state 11. This denotes the sequential behavior $S$ exhibited for trace $t$.
• Figure 2: High-level overview of MISH's workflow.
• Figure 3: Venn diagram for the faults detected by each algorithm on OCVN.
• Figure 4: Venn diagram for the faults detected by each algorithm on Scout-API.
• Figure 5: Average coverage over time on Features-Service