sec-certs: Examining the security certification practice for better vulnerability mitigation

TL;DR

The paper tackles the challenge of assessing vulnerability impact in security-certified products when data is scattered across unstructured Common Criteria artifacts. It introduces sec-certs, an open-source framework that automates collection, text extraction, and feature analysis of CC/FIPS certificates, maps CVEs to affected products via CPE matching, and builds a reference graph linking certificates. Through large-scale analysis and four case studies (ROCA, TPM-Fail, Minerva, Titan), it demonstrates improved coverage and faster vulnerability mitigation, and provides a web portal with weekly updates to aid stakeholders. The work highlights actionable recommendations to improve certification transparency, reproducibility, and data quality, and suggests future work using NLP to extract security-relevant information from artifacts and to extend to EMVCo artifacts.

Abstract

Products certified under security certification frameworks such as Common Criteria undergo significant scrutiny during the costly certification process. Yet, critical vulnerabilities, including private key recovery (ROCA, Minerva, TPM-Fail...), get discovered in certified products with high assurance levels. Furthermore, assessing which certified products are impacted by such vulnerabilities is complicated due to the large amount of unstructured certification-related data and unclear relationships between the certified products. To address these problems, we conducted a large-scale automated analysis of Common Criteria certificates. We trained unsupervised models to learn which vulnerabilities from NIST's National Vulnerability Database impact existing certified products and how certified products reference each other. Our tooling automates the analysis of tens of thousands of certification-related documents, extracting machine-readable features where manual analysis is unattainable. Further, we identify the security requirements that are associated with products being affected by fewer and less severe vulnerabilities. This indicates which aspects of certification correlate with higher security. We demonstrate how our tool can be used for better vulnerability mitigation on four case studies of known, high-profile vulnerabilities. All tools and continuously updated results are available at https://seccerts.org

Figures (3)

• Figure 1: A high-level overview of the sec-certs analysis framework.
• Figure 2: An example of mapping a certified product to a vulnerability that impacts it. While <vendor> fields in both CPEs match the vendor of the certified product, only the highlighted CPE has its <version> field appearing in the certified product title. Moreover, the string similarity between the lemmatized certificate title (turning the token "librarires" into its base form "library") yields a perfect match. Thus, the certified product matches the CPE and is likely impacted by the vulnerability.
• Figure 3: Disclosure dates of CVEs that affect certified products in relation to the certification date.