Forecasting the Maintained Score from the OpenSSF Scorecard for GitHub Repositories linked to PyPI libraries

TL;DR

OpenSSF Maintained is retrospective, reflecting only recent activity. The authors reconstruct three years of Maintained scores for 3,220 GitHub repos linked to top PyPI libraries (via PageRank) and cast forecasting as a multivariate time-series problem, evaluating VARMA, Random Forest, and LSTM across four representations. They find that aggregated representations—bucketed scores and trend types—are highly forecastable, while raw scores and trend slopes are more challenging, with simpler models often matching deep learning performance. The results suggest predictive maintenance signals can complement the Scorecard to enable proactive risk assessment and tooling integration.

Abstract

The OpenSSF Scorecard is widely used to assess the security posture of open-source software repositories, with the Maintained metric indicating recent development activity and helping identify potentially abandoned dependencies. However, this metric is inherently retrospective, reflecting only the past 90 days of activity and providing no insight into future maintenance, which limits its usefulness for proactive risk assessment. In this paper, we study to what extent future maintenance activity, as captured by the OpenSSF Maintained score, can be forecasted. We analyze 3,220 GitHub repositories associated with the top 1% most central PyPI libraries by PageRank and reconstruct historical Maintained scores over a three-year period. We formulate the task as multivariate time series forecasting and consider four target representations: raw scores, bucketed maintenance levels, numerical trend slopes, and categorical trend types. We compare a statistical model (VARMA), a machine learning model (Random Forest), and a deep learning model (LSTM) across training windows of 3-12 months and forecasting horizons of 1-6 months. Our results show that future maintenance activity can be predicted with meaningful accuracy, particularly for aggregated representations such as bucketed scores and trend types, achieving accuracies above 0.95 and 0.80, respectively. Simpler statistical and machine learning models perform on par with deep learning approaches, indicating that complex architectures are not required. These findings suggest that predictive modeling can effectively complement existing Scorecard metrics, enabling more proactive assessment of open-source maintenance risks.

Figures (4)

• Figure 1: Excerpt from the Scorecard for the numpy library
• Figure 2: Existing forecasting types
• Figure 3: Aggregated accuracy across model families and target representations, averaged over training windows, forecasting horizons, and temporally shifted sliding-windows
• Figure 4: Confusion matrices to asses misclassifications