Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Supporting Open Source Library Maintainers with Community-Based Analytics

Rachna Raj, Diego Elias Costa

TL;DR

The paper tackles the challenge that OSS library maintainers lack continuous feedback on how their APIs are used within the dependent ecosystem. It introduces community-based analytics comprising API Usage Analytics, Usage-Based API Test Coverage, and Community Test Coverage, and applies them to 10 popular Java libraries with 50 dependents each. The empirical study reveals that only a small fraction of APIs are broadly used, about three-quarters of used APIs are covered by tests, and community test coverage is modest but improvable via targeted tests; a practitioner survey indicates value in usage insights and testing-plan recommendations. The findings suggest a practical, ecosystem-aware approach to maintenance that can be integrated into existing workflows to prioritize API changes, testing, and evolution decisions, although limitations in generalizability and method-mapping are acknowledged. Overall, the work contributes a replication-ready framework and initial evidence that ecosystem-driven analytics can meaningfully support open-source library maintainers.

Abstract

Open-source software (OSS) is a pillar of modern software development. Its success depends on the dedication of maintainers who work constantly to keep their libraries stable, adapt to changing needs, and support a growing community. Yet, they receive little to no continuous feedback on how the projects that rely on their libraries actually use their APIs. We believe that gaining these insights can help maintainers make better decisions, such as refining testing strategies, understanding the impact of changes, and guiding the evolution of their libraries more effectively. We propose the use of community-based analytics to analyze how an OSS library is used across its dependent ecosystem. We conduct an empirical study of 10 popular Java libraries and each with their respective dependent ecosystem of 50 projects. Our results reveal that while library developers offer a wide range of API methods, only 16% on average are actively used by their dependent ecosystem. Moreover, only 74% of the used API methods are partially or fully covered by their library test suite. We propose two metrics to help developers evaluate their test suite according to the APIs used by their community, and we conduct a survey on open-source practitioners to assess the practical value of these insights in guiding maintenance decisions.

Towards Supporting Open Source Library Maintainers with Community-Based Analytics

TL;DR

The paper tackles the challenge that OSS library maintainers lack continuous feedback on how their APIs are used within the dependent ecosystem. It introduces community-based analytics comprising API Usage Analytics, Usage-Based API Test Coverage, and Community Test Coverage, and applies them to 10 popular Java libraries with 50 dependents each. The empirical study reveals that only a small fraction of APIs are broadly used, about three-quarters of used APIs are covered by tests, and community test coverage is modest but improvable via targeted tests; a practitioner survey indicates value in usage insights and testing-plan recommendations. The findings suggest a practical, ecosystem-aware approach to maintenance that can be integrated into existing workflows to prioritize API changes, testing, and evolution decisions, although limitations in generalizability and method-mapping are acknowledged. Overall, the work contributes a replication-ready framework and initial evidence that ecosystem-driven analytics can meaningfully support open-source library maintainers.

Abstract

Open-source software (OSS) is a pillar of modern software development. Its success depends on the dedication of maintainers who work constantly to keep their libraries stable, adapt to changing needs, and support a growing community. Yet, they receive little to no continuous feedback on how the projects that rely on their libraries actually use their APIs. We believe that gaining these insights can help maintainers make better decisions, such as refining testing strategies, understanding the impact of changes, and guiding the evolution of their libraries more effectively. We propose the use of community-based analytics to analyze how an OSS library is used across its dependent ecosystem. We conduct an empirical study of 10 popular Java libraries and each with their respective dependent ecosystem of 50 projects. Our results reveal that while library developers offer a wide range of API methods, only 16% on average are actively used by their dependent ecosystem. Moreover, only 74% of the used API methods are partially or fully covered by their library test suite. We propose two metrics to help developers evaluate their test suite according to the APIs used by their community, and we conduct a survey on open-source practitioners to assess the practical value of these insights in guiding maintenance decisions.
Paper Structure (24 sections, 3 equations, 4 figures, 6 tables)

This paper contains 24 sections, 3 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Data Selection
  5. Extracting API Usage Analytics
  6. Measuring Test Coverage
  7. Usage-Based API Test Coverage
  8. Community Test Coverage
  9. Surveying Open Source Library Maintainers
  10. API Usage Analytics
  11. How are libraries used by their dependent ecosystem?
  12. Is API Usage Analytics useful for maintainers?
  13. Usage-Based API Test Coverage
  14. How often are used API methods covered by tests?
  15. Is Usage-Based API Test Coverage useful?
  16. ...and 9 more sections

Figures (4)

  • Figure 1: Overview Diagram of this project
  • Figure 2: Distribution of dependent's repository metrics across a representative datasets for 10 candidate libraries.
  • Figure 3: Percentage of the total available API methods used by the 50 dependent projects. We highlight the mean percentage across libraries with a vertical dotted mark.
  • Figure 4: Survey questions and results from the survey of 20 OSS contributors and maintainers.