COTS: Connected OpenAPI Test Synthesis for RESTful Applications

TL;DR

The paper addresses the difficulty of detecting logic-based faults in RESTful APIs, where fully automatic test generators and handcrafted tests have complementary limitations. It proposes a model-driven approach that encodes API dependencies and stateful interactions in COpenAPI, a DSL inspired by binary session types, and uses the COTS tool to generate and execute tests against OpenAPI-described SUTs. The methodology comprises modelling, automated test generation, and analysis phases to identify deviations from the model and report faults. Evaluation on nine open-source applications uncovered 25 faults (10 logic-based and 15 systematic) and showed that COTS achieves comparable or higher code coverage than handcrafted REST tests and the Morest tool, often with much smaller COpenAPI models.

Abstract

We present a novel model-driven approach for testing RESTful applications. We introduce a (i) domain-specific language for OpenAPI specifications and (ii) a tool to support our methodology. Our DSL is inspired by session types and enables the modelling of communication protocols between a REST client and server. Our tool, dubbed COTS, generates (randomised) model-based test executions and reports software defects. We evaluate the effectiveness of our approach by applying it to test several open source applications. Our findings indicate that our methodology can identify nuanced defects in REST APIs and achieve comparable or superior code coverage when compared to much larger handcrafted test suites.