Do Chase Your Tail! Missing Key Aspects Augmentation in Textual Vulnerability Descriptions of Long-tail Software through Feature Inference

TL;DR

This paper tackles missing key aspects in TVDs for long-tail software by introducing a software feature inference framework that combines standardized software naming via government CVE mappings, CWE-based expansion to diversify training examples, and LLM-driven generation augmented with an NLI-based hallucination filter and wiki-grounded background knowledge. The approach yields significant improvements in generating and selecting missing key aspects, with long-tail software performance rising from $0.27$ to $0.56$ in accuracy and non-long-tail performance also improving, while enabling stronger downstream task performance such as CVSS and CWE prediction. Key innovations include a Software-CVE Mapping Database, CWE-driven retrieval and clustering for representative in-context examples, an NLI-based software-feature correlation model, and a per-software hallucination-detection paradigm grounded with DSSM embeddings and Wikipedia content. The framework demonstrates robust generalization across datasets (CVE, NVD, NVD*) and tasks, offering practical benefits for vulnerability analysis and software maintenance in real-world settings where long-tail software are prevalent and TVDs are incomplete.

Abstract

Augmenting missing key aspects in Textual Vulnerability Descriptions (TVDs) is crucial for effective vulnerability analysis. For instance, in TVDs, key aspects include Attack Vector, Vulnerability Type, among others. These key aspects help security engineers understand and address the vulnerability in a timely manner. For software with a large user base (non-long-tail software), augmenting these missing key aspects has significantly advanced vulnerability analysis and software security research. However, software instances with a limited user base (long-tail software) often get overlooked due to inconsistency software names, TVD limited avaliability, and domain-specific jargon, which complicates vulnerability analysis and software repairs. In this paper, we introduce a novel software feature inference framework designed to augment the missing key aspects of TVDs for long-tail software. Firstly, we tackle the issue of non-standard software names found in community-maintained vulnerability databases by cross-referencing government databases with Common Vulnerabilities and Exposures (CVEs). Next, we employ Large Language Models (LLMs) to generate the missing key aspects. However, the limited availability of historical TVDs restricts the variety of examples. To overcome this limitation, we utilize the Common Weakness Enumeration (CWE) to classify all TVDs and select cluster centers as representative examples. To ensure accuracy, we present Natural Language Inference (NLI) models specifically designed for long-tail software. These models identify and eliminate incorrect responses. Additionally, we use a wiki repository to provide explanations for proprietary terms.

Figures (10)

• Figure 1: In the coordinate axis, the x-axis represents the software instance represented by IDs, and the y-axis represents the number of TVDs each software has. For example, Windows, id=17, and has 509 TVDs. For CVE-2002-0679 (Microsoft Windows) and CVE-2016-0058 (ToolTalk RPC), we omitted the Attack Vector information from TVDs. Microsoft Windows has 509 TVDs in the NVD, whilst ToolTalk RPC only has 31 TVDs. To predict the missing key aspect (Attack Vector) based on the rest of TVD's key aspects, none of the existing methods can successfully complete the task for ToolTalk RPC. For Microsoft Windows, the vulnerability stems from file operations during runtime. It involves supporting a file storage system and facilitating system modification, which could enable attacks via manipulated files. For ToolTalk RPC, a service software reliant on incoming parameters, its vulnerability to parameter-based attacks is exacerbated by the absence of built-in file transmission support. Therefore, for SOTA predicting the same Attack Vector, it is applicable to Windows but not to ToolTalk RPC.
• Figure 2: Accuracy on classification task. The Impact aspect has a value of 0, as PMA deems the missing rate of Impact low and unnecessary to predict.
• Figure 3: Performance of the generative prediction task on missing key aspect augmentation.
• Figure 4: Approach overview: First, to construct software-CVE mapping database, we leverage government databases to resolve software naming references. Second, we utilize the CWE to enhance the diversity of historical TVDs associated with long-tail software. Lastly, in the face of the heightened risk of hallucination in LLMs due to the limited long-tail software knowledge stored, we employ software feature inference to mitigate this issue.
• Figure 5: Software naming coreference resolution