How Quickly Do Development Teams Update Their Vulnerable Dependencies?

TL;DR

This work introduces two novel dependency update metrics, $MTTU_{dep}$ and $MTTR_{dep}$, to quantify how quickly development teams update dependencies and remediate vulnerable ones. By partitioning each <package,dependency> relation into interval-based windows and applying exponential weighting, the authors address limitations of prior measures such as floating-version handling and recency emphasis. An extensive empirical study across npm, PyPI, and Cargo shows most packages update dependencies quickly, but vulnerability data is sparse, limiting full MTTR assessment; $MTTU_{dep}$ can serve as a partial proxy for MTTR_{dep} with caution. The findings offer actionable guidance for developers, researchers, and tool builders aiming to improve software supply chain security and maintenance practices while highlighting human and policy factors that influence dependency updatedness.

Abstract

Industry practitioners are increasingly concerned with software that contains vulnerable versions of third-party dependencies that are included both directly and transitively. To address this problem, projects are encouraged to both (a)~quickly update to non-vulnerable versions of dependencies and (b)~be mindful of the update practices of the dependencies they choose to use. To this end, researchers have proposed metrics to measure the responsiveness of the development teams of the packages in keeping their dependencies updated: Mean-Time-To-Update (MTTU) and Mean-Time-To-Remediate (MTTR). While MTTU covers all dependencies, MTTR quantifies the time needed for a package to update its vulnerable dependencies. However, existing metrics fail to capture important nuances, such as considering floating versions and prioritizing recent updates, leading to inaccurate reflections of a development team's update practices. \textit{The goal of this study is to aid practitioners in understanding how quickly packages update their dependencies.} We propose two novel metrics, Mean-Time-To-Update for dependencies (MTTU) and Mean-Time-To-Remediate for vulnerable dependencies (MTTR), that overcome the limitations of existing metrics. We conduct an empirical study using $163,207$ packages in npm ($117,129$), PyPI ($42,777$), and Cargo ($3,301$) and characterize how the ecosystems differ in MTTU and MTTR, as well as what package characteristics influence MTTU and MTTR. We found that most packages have a relatively fast dependency update practice. We further study whether MTTU can be used as a proxy for MTTR when sufficient vulnerability data is not available. As we did not find enough statistical evidence for a strong proxy, our findings suggest that MTTU could only be partially used (may be used but with caution) as a proxy for MTTR when vulnerability data is not available.

Figures (4)

• Figure 1: Illustration of tLag, and MTTUdep/MTTRdep calculation using codemod-cli's dependency relationship with simple-git. Blue time quantum indicates outdated dependency, red time quantum indicates outdated and vulnerable dependency, and yellow quantum indicates updated dependency.
• Figure 2: ${MTTU}_{dep}$
• Figure 3: ${MTTR}_{dep}$
• Figure 4: Correlation matrix on packages' characteristics and MTTUdep and MTTRdep values.