Signing in Four Public Software Package Registries: Quantity, Quality, and Influencing Factors

TL;DR

This work measures software signing practices across four major public registries (PyPI, Maven Central, Docker Hub, Hugging Face) to quantify the prevalence and quality of maintainer signatures and understand driving factors. Using a five-stage methodology and a quasi-experimental design, it compares registries with different signing policies and tooling, and tests four incentives-based hypotheses. Key findings show that mandatory signing yields high signing quantity, dedicated tooling yields perfect signing quality, and the first signature significantly lowers the barrier to subsequent signing, while cybersecurity events have limited impact. The results highlight the pivotal role of registry operators in shaping secure software provenance and provide concrete guidance for policy and tooling investments to improve supply-chain security.

Abstract

Many software applications incorporate open-source third-party packages distributed by public package registries. Guaranteeing authorship along this supply chain is a challenge. Package maintainers can guarantee package authorship through software signing. However, it is unclear how common this practice is, and whether the resulting signatures are created properly. Prior work has provided raw data on registry signing practices, but only measured single platforms, did not consider quality, did not consider time, and did not assess factors that may influence signing. We do not have up-to-date measurements of signing practices nor do we know the quality of existing signatures. Furthermore, we lack a comprehensive understanding of factors that influence signing adoption. This study addresses this gap. We provide measurements across three kinds of package registries: traditional software (Maven, PyPI), container images (DockerHub), and machine learning models (Hugging Face). For each registry, we describe the nature of the signed artifacts as well as the current quantity and quality of signatures. Then, we examine longitudinal trends in signing practices. Finally, we use a quasi-experiment to estimate the effect that various factors had on software signing practices. To summarize our findings: (1) mandating signature adoption improves the quantity of signatures; (2) providing dedicated tooling improves the quality of signing; (3) getting started is the hard part -- once a maintainer begins to sign, they tend to continue doing so; and (4) although many supply chain attacks are mitigable via signing, signing adoption is primarily affected by registry policy rather than by public knowledge of attacks, new engineering standards, etc. These findings highlight the importance of software package registry managers and signing infrastructure.

Figures (10)

• Figure 1: Maintainers create software packages and signers create keys which are used to create a signature. Each of these artifacts are published to a registry. Depending on ecosystem, the registries and the actors may or may not be separate. Users fetch these artifacts and can check signatures using infrastructure-specific tooling. This creates a verified package. Red and orange numbers indicate the failure modes described in \ref{['sec:signing_process']}. Red numbers indicate discernible failures. The orange numbers (modes 1, 4, and 8) are not distinguishable from one another by an external audit --- when keys are missing, we cannot determine whether they were never created (mode 1), were not published (mode 4), or were undiscoverable by us (mode 8).
• Figure 2: Incentives influence how maintainers adopt software signing. The maintainer decides weather or not to create a signature. If a maintainer decides to create a signature, they can either follow the signing process (i.e.,\ref{['fig:good_signing']}) correctly or not. Correctly following the signing process results in a good signature but incorrectly following the process results in a signing failure.
• Figure 3: First, we select package registries that represent a range of software types and signing policies. Our selected registries include PyPI, Maven Central, Docker Hub, and Hugging Face. Next, we collect a list of packages for each platform. Then, we filter the list of packages to a sample of packages for each platform. On the remaining packages, we measure the quality and quantity of signatures. Finally, we use these measurements to evaluate factors influencing adoption.
• Figure 4: Quantity of signed artifacts over time. Axes are time (monthly increments) and the percentage of signed artifacts per registry.
• Figure 5: Quality of signed artifacts over time. X-axis shows time (monthly increments). Y-axis shows percentage of signatures with good status.