Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uncovering Scientific Software Sustainability through Community Engagement and Software Quality Metrics

Sharif Ahmed, Addi Malviya Thakur, Gregory R. Watson, Nasir U. Eisty

TL;DR

The paper addresses the challenge of sustaining scientific open-source software (Sci-OSS) by identifying two core drivers—community engagement and software quality—and mapping them to repository metrics drawn from GitHub data. It introduces the Software SusTainability Graph (STG), a multimodal visualization that consolidates 31 metrics across 18 leads and 46 components to display current and evolving sustainability signals for ten Sci-OSS projects. Through visual analysis, non-parametric statistics, and NLP-driven conversation analysis, the authors demonstrate that project-specific feedback and dynamics influence sustainability, with limited evidence for domain-wide patterns. The work provides a dataset, a compact visualization approach, and methodological foundations that researchers, funders, and developers can use to assess and support long-term Sci-OSS sustainability, and it outlines future work to broaden metric coverage and integrate these tools into practice.

Abstract

Scientific open-source software (Sci-OSS) projects are critical for advancing research, yet sustaining these projects long-term remains a major challenge. This paper explores the sustainability of Sci-OSS hosted on GitHub, focusing on two factors drawn from stewardship organizations: community engagement and software quality. We map sustainability to repository metrics from the literature and mined data from ten prominent Sci-OSS projects. A multimodal analysis of these projects led us to a novel visualization technique, providing a robust way to display both current and evolving software metrics over time, replacing multiple traditional visualizations with one. Additionally, our statistical analysis shows that even similar-domain projects sustain themselves differently. Natural language analysis supports claims from the literature, highlighting that project-specific feedback plays a key role in maintaining software quality. Our visualization and analysis methods offer researchers, funders, and developers key insights into long-term software sustainability.

Uncovering Scientific Software Sustainability through Community Engagement and Software Quality Metrics

TL;DR

The paper addresses the challenge of sustaining scientific open-source software (Sci-OSS) by identifying two core drivers—community engagement and software quality—and mapping them to repository metrics drawn from GitHub data. It introduces the Software SusTainability Graph (STG), a multimodal visualization that consolidates 31 metrics across 18 leads and 46 components to display current and evolving sustainability signals for ten Sci-OSS projects. Through visual analysis, non-parametric statistics, and NLP-driven conversation analysis, the authors demonstrate that project-specific feedback and dynamics influence sustainability, with limited evidence for domain-wide patterns. The work provides a dataset, a compact visualization approach, and methodological foundations that researchers, funders, and developers can use to assess and support long-term Sci-OSS sustainability, and it outlines future work to broaden metric coverage and integrate these tools into practice.

Abstract

Scientific open-source software (Sci-OSS) projects are critical for advancing research, yet sustaining these projects long-term remains a major challenge. This paper explores the sustainability of Sci-OSS hosted on GitHub, focusing on two factors drawn from stewardship organizations: community engagement and software quality. We map sustainability to repository metrics from the literature and mined data from ten prominent Sci-OSS projects. A multimodal analysis of these projects led us to a novel visualization technique, providing a robust way to display both current and evolving software metrics over time, replacing multiple traditional visualizations with one. Additionally, our statistical analysis shows that even similar-domain projects sustain themselves differently. Natural language analysis supports claims from the literature, highlighting that project-specific feedback plays a key role in maintaining software quality. Our visualization and analysis methods offer researchers, funders, and developers key insights into long-term software sustainability.

Paper Structure

This paper contains 30 sections, 1 equation, 7 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. A. Factors and Metrics for Sustainability
  4. Leading factors
  5. Metrics
  6. B. Data Collection
  7. C. Data Processing
  8. Repository Metrics
  9. Temporal Metrics
  10. Model-computed Metrics
  11. Community Belonging and Engagement (CBE) Metrics
  12. D. Data Analysis
  13. Visual Analysis
  14. Statistical Analysis
  15. Conversation Analysis
  16. ...and 15 more sections

Figures (7)

  • Figure 1: Study methodology: identified sustainability-related software metrics from the literature (left), extracted these metrics from 10 open-source scientific projects (middle), and analyzed the data by developing a new visualization technique (right).
  • Figure 2: Metrics with Traditional Visualizations
  • Figure 3: Our Proposed Software SusTainability Graph (STG) showing P8. STG structure is outlined in TABLE \ref{['tbl_stg_struct']}
  • Figure 4: STG interpretation
  • Figure 5: Sustainability Graph of Studied 10 Projects
  • ...and 2 more figures