Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lightweight Syntactic API Usage Analysis with UCov

Gustave Monce, Thomas Couturou, Yasmine Hamdaoui, Thomas Degueule, Jean-Rémy Falleri

TL;DR

The paper introduces syntactic usage models (SUMs) and syntactic usage footprints (SUFs) as a lightweight, formal framework to capture the full spectrum of interactions allowed by an API and those actually exercised by client code. It presents UCov, a Java-based static-analysis tool that derives SUMs from library code and SUFs from client code to compute usage-coverage metrics, enabling library maintainers to identify under-tested or undocumented interactions and to assess the impact of API changes. Through an exploratory case study of jsoup, commons-cli, and spark, the authors demonstrate how SUMs/SUFs reveal design patterns, usage profiles, and practical gaps between documented examples, tests, and real-world usage. The work highlights implications for API design, testing, and evolution, and outlines future directions for broader language support and automated regression/testing based on complete SUMs.

Abstract

Designing an effective API is essential for library developers as it is the lens through which clients will judge its usability and benefits, as well as the main friction point when the library evolves. Despite its importance, defining the boundaries of an API is a challenging task, mainly due to the diverse mechanisms provided by programming languages that have non-trivial interplays. In this paper, we present a novel conceptual framework designed to assist library maintainers in understanding the interactions allowed by their APIs via the use of syntactic usage models. These customizable models enable library maintainers to improve their design ahead of release, reducing friction during evolution. The complementary syntactic usage footprints and coverage scores, inferred from client code using the API (e.g., documentation samples, tests, third-party clients), enable developers to understand in-the-wild uses of their APIs and to reflect on the adequacy of their tests and documentation. We implement these models for Java libraries in a new tool UCov and demonstrate its capabilities on three libraries exhibiting diverse styles of interaction: jsoup, commons-cli, and spark. Our exploratory case study shows that UCov provides valuable information regarding API design and fine-grained analysis of client code to identify under-tested and under-documented library code.

Lightweight Syntactic API Usage Analysis with UCov

TL;DR

The paper introduces syntactic usage models (SUMs) and syntactic usage footprints (SUFs) as a lightweight, formal framework to capture the full spectrum of interactions allowed by an API and those actually exercised by client code. It presents UCov, a Java-based static-analysis tool that derives SUMs from library code and SUFs from client code to compute usage-coverage metrics, enabling library maintainers to identify under-tested or undocumented interactions and to assess the impact of API changes. Through an exploratory case study of jsoup, commons-cli, and spark, the authors demonstrate how SUMs/SUFs reveal design patterns, usage profiles, and practical gaps between documented examples, tests, and real-world usage. The work highlights implications for API design, testing, and evolution, and outlines future directions for broader language support and automated regression/testing based on complete SUMs.

Abstract

Designing an effective API is essential for library developers as it is the lens through which clients will judge its usability and benefits, as well as the main friction point when the library evolves. Despite its importance, defining the boundaries of an API is a challenging task, mainly due to the diverse mechanisms provided by programming languages that have non-trivial interplays. In this paper, we present a novel conceptual framework designed to assist library maintainers in understanding the interactions allowed by their APIs via the use of syntactic usage models. These customizable models enable library maintainers to improve their design ahead of release, reducing friction during evolution. The complementary syntactic usage footprints and coverage scores, inferred from client code using the API (e.g., documentation samples, tests, third-party clients), enable developers to understand in-the-wild uses of their APIs and to reflect on the adequacy of their tests and documentation. We implement these models for Java libraries in a new tool UCov and demonstrate its capabilities on three libraries exhibiting diverse styles of interaction: jsoup, commons-cli, and spark. Our exploratory case study shows that UCov provides valuable information regarding API design and fine-grained analysis of client code to identify under-tested and under-documented library code.
Paper Structure (30 sections, 3 figures, 5 tables)

This paper contains 30 sections, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Syntactic API Usage
  4. Syntactic Usage Model (SUM)
  5. Syntactic Usage Footprint (SUF)
  6. Usage Coverage and Metrics
  7. UCov: Java Syntactic Usage Analysis
  8. Exported Symbols
  9. Supported API Symbol Uses
  10. Implementation
  11. Exploratory Case Study
  12. Subject Libraries
  13. Interaction styles
  14. Client code
  15. Selected libraries
  16. ...and 15 more sections

Figures (3)

  • Figure 1: UCov parses and analyses Java code to build SUM and SUF models. Each cross symbol ($\times$) represents one possible use of an API symbol.
  • Figure 2: Usage profiles of the three libraries commons-cli, jsoup, and spark
  • Figure 3: UpSet plots 2014_infovis_upset depicting the common and unique uses between third-party clients, tests, and samples for jsoup, commons-cli, and spark