Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OpenAI for OpenAPI: Automated generation of REST API specification via LLMs

Hao Chen, Yunchun Li, Chen Chen, Fengxu Lin, Wei Li

TL;DR

This work tackles the challenge of generating OpenAPI specifications (OAS) for REST APIs across diverse languages and frameworks by introducing OOPS, a technology-agnostic, LLM-driven static analysis method. OOPS combines server-side technology analysis, endpoint method extraction, and OpenAPI specification generation within a two-stage, multi-agent workflow, using an API dependency graph to address LLM context limits and a self-refine process to reduce hallucinations. Empirical evaluation on 12 REST APIs (5 languages, 8 frameworks) shows OOPS achieves near-perfect endpoint-method inference (average F1 ≈ 0.98) and high precision/recall across request parameters, responses, and constraints, outperforming baselines like APICARV, ExpressO, and Respector. The approach demonstrates strong generalizability and cost-efficiency (≈$0.41 per OAS) and sets the stage for broader automation of API documentation, testing, and security tasks with LLM-based tooling.

Abstract

REST APIs, based on the REpresentational State Transfer (REST) architecture, are the primary type of Web API. The OpenAPI Specification (OAS) serves as the de facto standard for describing REST APIs and is crucial for multiple software engineering tasks. However, developers face challenges in writing and maintaining OAS. Although static analysis shows potential for OAS generation, it is limited to specific programming languages and development frameworks. The powerful code understanding capabilities of LLMs offer new opportunities for OAS generation, yet they are constrained by context limitations and hallucinations. To address these challenges, we propose the OpenAI OpenAPI Project Scanner (OOPS), the first technology-agnostic LLM-based static analysis method for OAS generation, requiring fewer technology-specific rules and less human expert intervention. OOPS is implemented as an LLM agent workflow comprising two key steps: endpoint method extraction and OAS generation. By constructing an API dependency graph, it establishes necessary file associations to address LLMs' context limitations. Through multi-stage generation and self-refine, it mitigates both syntactic and semantic hallucinations during OAS generation. We evaluated OOPS on 12 real-world REST APIs spanning 5 programming languages and 8 development frameworks. Experimental results demonstrate that OOPS accurately generates high-quality OAS for REST APIs implemented with diverse technologies, achieving an average F1-score exceeding 98% for endpoint method inference, 97% for both request parameter and response inference, and 92% for parameter constraint inference. The input tokens average below 5.6K with a maximum of 16.2K, while the output tokens average below 0.9K with a maximum of 7.7K.

OpenAI for OpenAPI: Automated generation of REST API specification via LLMs

TL;DR

This work tackles the challenge of generating OpenAPI specifications (OAS) for REST APIs across diverse languages and frameworks by introducing OOPS, a technology-agnostic, LLM-driven static analysis method. OOPS combines server-side technology analysis, endpoint method extraction, and OpenAPI specification generation within a two-stage, multi-agent workflow, using an API dependency graph to address LLM context limits and a self-refine process to reduce hallucinations. Empirical evaluation on 12 REST APIs (5 languages, 8 frameworks) shows OOPS achieves near-perfect endpoint-method inference (average F1 ≈ 0.98) and high precision/recall across request parameters, responses, and constraints, outperforming baselines like APICARV, ExpressO, and Respector. The approach demonstrates strong generalizability and cost-efficiency (≈$0.41 per OAS) and sets the stage for broader automation of API documentation, testing, and security tasks with LLM-based tooling.

Abstract

REST APIs, based on the REpresentational State Transfer (REST) architecture, are the primary type of Web API. The OpenAPI Specification (OAS) serves as the de facto standard for describing REST APIs and is crucial for multiple software engineering tasks. However, developers face challenges in writing and maintaining OAS. Although static analysis shows potential for OAS generation, it is limited to specific programming languages and development frameworks. The powerful code understanding capabilities of LLMs offer new opportunities for OAS generation, yet they are constrained by context limitations and hallucinations. To address these challenges, we propose the OpenAI OpenAPI Project Scanner (OOPS), the first technology-agnostic LLM-based static analysis method for OAS generation, requiring fewer technology-specific rules and less human expert intervention. OOPS is implemented as an LLM agent workflow comprising two key steps: endpoint method extraction and OAS generation. By constructing an API dependency graph, it establishes necessary file associations to address LLMs' context limitations. Through multi-stage generation and self-refine, it mitigates both syntactic and semantic hallucinations during OAS generation. We evaluated OOPS on 12 real-world REST APIs spanning 5 programming languages and 8 development frameworks. Experimental results demonstrate that OOPS accurately generates high-quality OAS for REST APIs implemented with diverse technologies, achieving an average F1-score exceeding 98% for endpoint method inference, 97% for both request parameter and response inference, and 92% for parameter constraint inference. The input tokens average below 5.6K with a maximum of 16.2K, while the output tokens average below 0.9K with a maximum of 7.7K.
Paper Structure (29 sections, 5 figures, 7 tables, 2 algorithms)

This paper contains 29 sections, 5 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background and motivation
  3. REST API and OAS
  4. Research motivation
  5. Approach
  6. Overview
  7. Endpoint method extraction
  8. OpenAPI Specification generation
  9. Experimental methodology
  10. Research questions
  11. Datasets
  12. Baselines
  13. Performance metrics
  14. Implementation details
  15. Experimental results
  16. ...and 14 more sections

Figures (5)

  • Figure 1: Example of a Swagger 2.0 specification.
  • Figure 2: Example of an OpenAPI 3.0 specification.
  • Figure 3: Overview of the OOPS approach.
  • Figure 4: An example of an API dependency graph.
  • Figure 5: Common violations of the OAS schema observed in LLM-generated outputs, with incorrect output on the left and correct output on the right.