Real-VulLLM: An LLM Based Assessment Framework in the Wild

TL;DR

This work addresses the challenge of evaluating LLMs for vulnerability detection and reasoning in real-world, unknown-context scenarios. It proposes Real-VulLLM, a framework combining a NVD-derived CVE vector store with retrieval-augmented prompts across four strategies to assess multiple LLMs. A hybrid evaluator jointly measures prediction accuracy, reasoning quality, and partial correctness, enabling richer judgment than binary metrics. Experiments across five major LLMs show that knowledge augmentation improves performance, with decomposition and plan-and-solve prompts offering the strongest gains, though generalization to unseen CVEs remains limited. The framework offers a practical path toward safer, more reliable deployment of LLMs in software security workflows by emphasizing real-world context, structured reasoning, and automated, multi-faceted evaluation.

Abstract

Artificial Intelligence (AI) and more specifically Large Language Models (LLMs) have demonstrated exceptional progress in multiple areas including software engineering, however, their capability for vulnerability detection in the wild scenario and its corresponding reasoning remains underexplored. Prompting pre-trained LLMs in an effective way offers a computationally effective and scalable solution. Our contributions are (i)varied prompt designs for vulnerability detection and its corresponding reasoning in the wild. (ii)a real-world vector data store constructed from the National Vulnerability Database, that will provide real time context to vulnerability detection framework, and (iii)a scoring measure for combined measurement of accuracy and reasoning quality. Our contribution aims to examine whether LLMs are ready for wild deployment, thus enabling the reliable use of LLMs stronger for the development of secure software's.

Figures (10)

• Figure 1: Illustrative example of LLM robustness limitations . In (a), the vulnerability is correctly flagged; in (b), after adding a harmless library include or variable renaming, the vulnerability persists but models often misclassify it as secure.
• Figure 2: Prompt Template: The LLM is provided with a structured instruction, a system prompt assigning role of security expert, a user-defined query, and a real-world code snippet. It outputs prediction and reason. Prediction identifies the vulnerability and reason present the cause/justification.
• Figure 3: Related work in Software Vulnerability Detection
• Figure 4: Architecture of Real-VulLLM
• Figure 5: Structured prompt template used for querying LLMs