Fuzzing REST APIs in Industry: Necessary Features and Open Problems

Abstract

REST APIs are widely used in industry, in all different kinds of domains. An example is Volkswagen AG, a German automobile manufacturer. Established testing approaches for REST APIs are time consuming, and require expertise from professional test engineers. Due to its cost and importance, in the scientific literature several approaches have been proposed to automatically test REST APIs. The open-source, search-based fuzzer EvoMaster is one of such tools proposed in the academic literature. However, how academic prototypes can be integrated in industry and have real impact to software engineering practice requires more investigation. In this paper, we report on our experience in using EvoMaster at Volkswagen AG, as an EvoMaster user from 2023 to 2026. We share our learnt lessons, and discuss several features needed to be implemented in EvoMaster to make its use in an industrial context successful. Feedback about value in industrial setups of EvoMaster was given from Volkswagen AG about 4 APIs. Additionally, a user study was conducted involving 11 testing specialists from 4 different companies. We further identify several real-world research challenges that still need to be solved.

Figures (10)

• Figure 1: Extract from an OpenAPI schema of an artificial API example, with a links definition.
• Figure 2: Example of generated test showing the dynamic use of a link, based on the OpenAPI schema defined in Figure \ref{['fig:schema']}.
• Figure 3: Faulty definition of endpoint, based on schema from Figure \ref{['fig:schema']}.
• Figure 4: Example of TOML configuration file for authentication using tokens extracted from a login endpoint, and that can then be sent as Bearer in the HTTP Authorization header.
• Figure 5: Example of generated test showing how authentication tokens can be dynamically retrieved and used in following HTTP calls.