Elevating Software Trust: Unveiling and Quantifying the Risk Landscape
Sarah Ali Siddiqui, Chandra Thapa, Rayne Holland, Wei Shao, Seyit Camtepe
TL;DR
SAFER reframes software risk as a trust-centric, dynamic problem, incorporating developer, publisher, and user factors along with code and dependency attributes. It uses five parameter-selection steps and a Logic Scoring of Preferences (LSP) aggregation to compute actor-based risks ($R^{\text{DEV}}$, $R^{\text{PB}}$, $R^{\text{UR}}$) and a final risk $R^{\text{F}}$, further adjusted by a context-dependent penalty to yield $R^{\text{FP}}$ within four bands. The framework emphasizes data-driven, time-aware weights ($w^{\text{DEV}}$, $w^{\text{PB}}$, $w^{\text{UR}}$) with a sigmoid mapping to ensure comparability, and it supports tunability (e.g., code dependencies, risk bands, and penalties) for organizational needs. Empirical evaluation on a 9000-sample dataset shows SAFER reduces subjectivity and better captures evolving threat factors than existing tools like the OWASP risk calculator and OpenSSF scorecard, with visual analyses illustrating parameter sensitivity and dynamicity. The work outlines practical implications for informed decision-making, risk prioritization, and trust-aware adoption of software, and proposes future work including simulators and reinforcement learning to enhance risk quantification and validation.
Abstract
Considering the ever-evolving threat landscape and rapid changes in software development, we propose a risk assessment framework called SAFER (Software Analysis Framework for Evaluating Risk). This framework is based on the necessity of a dynamic, data-driven, and adaptable process to quantify security risk in the software supply chain. Usually, when formulating such frameworks, static pre-defined weights are assigned to reflect the impact of each contributing parameter while aggregating these individual parameters to compute resulting security risk scores. This leads to inflexibility, a lack of adaptability, and reduced accuracy, making them unsuitable for the changing nature of the digital world. We adopt a novel perspective by examining security risk through the lens of trust and incorporating the human aspect. Moreover, we quantify security risk associated with individual software by assessing and formulating risk elements quantitatively and exploring dynamic data-driven weight assignment. This enhances the sensitivity of the framework to cater to the evolving security risk factors associated with software development and the different actors involved in the entire process. The devised framework is tested through a dataset containing 9000 samples, comprehensive scenarios, assessments, and expert opinions. Furthermore, a comparison between scores computed by the OpenSSF scorecard, OWASP risk calculator, and the proposed SAFER framework has also been presented. The results suggest that SAFER mitigates subjectivity and yields dynamic data-driven weights as well as security risk scores.