Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

vEcho: A Paradigm Shift from Vulnerability Verification to Proactive Discovery with Large Language Models

Mingcheng Jiang, Jiancheng Huang, Jiangfei Wang, Zhengzhu Xie, Nan Fang, Guang Cheng, Xiaoyan Hu, Hua Wu

Abstract

Static Application Security Testing (SAST) tools often suffer from high false positive rates, leading to alert fatigue that consumes valuable auditing resources. Recent efforts leveraging Large Language Models (LLMs) as filters offer limited improvements; however, these methods treat LLMs as passive, stateless classifiers, which lack project-wide context and the ability to learn from analyses to discover unknown, similar vulnerabilities.In this paper, we propose vEcho, a novel framework that transforms the LLM from a passive filter into a virtual security expert capable of learning, memory, and reasoning. vEcho equips its core reasoning engine with a robust developer tool suite for deep, context-aware verification. More importantly, we introduce a novel Echoic Vulnerability Propagation (EVP) mechanism. Driven by a Cognitive Memory Module that simulates human learning, EVP enables vEcho to learn from verified vulnerabilities and proactively infer unknown, analogous flaws, achieving a paradigm shift from passive verification to active discovery.Extensive experiments on the CWE-Bench-Java dataset demonstrate vEcho's dual advantages over the state-of-the-art baseline, IRIS. Specifically, vEcho achieves a 65% detection rate, marking a 41.8% relative improvement over IRIS's 45.83%. Crucially, it simultaneously addresses alert fatigue by reducing the false positive rate to 59.78%, a 28.3% relative reduction from IRIS's 84.82%. Furthermore, vEcho proactively identified 37 additional known vulnerabilities beyond the 120 documented in the dataset, and has discovered 51 novel 0-day vulnerabilities in open-source projects.

vEcho: A Paradigm Shift from Vulnerability Verification to Proactive Discovery with Large Language Models

Abstract

Static Application Security Testing (SAST) tools often suffer from high false positive rates, leading to alert fatigue that consumes valuable auditing resources. Recent efforts leveraging Large Language Models (LLMs) as filters offer limited improvements; however, these methods treat LLMs as passive, stateless classifiers, which lack project-wide context and the ability to learn from analyses to discover unknown, similar vulnerabilities.In this paper, we propose vEcho, a novel framework that transforms the LLM from a passive filter into a virtual security expert capable of learning, memory, and reasoning. vEcho equips its core reasoning engine with a robust developer tool suite for deep, context-aware verification. More importantly, we introduce a novel Echoic Vulnerability Propagation (EVP) mechanism. Driven by a Cognitive Memory Module that simulates human learning, EVP enables vEcho to learn from verified vulnerabilities and proactively infer unknown, analogous flaws, achieving a paradigm shift from passive verification to active discovery.Extensive experiments on the CWE-Bench-Java dataset demonstrate vEcho's dual advantages over the state-of-the-art baseline, IRIS. Specifically, vEcho achieves a 65% detection rate, marking a 41.8% relative improvement over IRIS's 45.83%. Crucially, it simultaneously addresses alert fatigue by reducing the false positive rate to 59.78%, a 28.3% relative reduction from IRIS's 84.82%. Furthermore, vEcho proactively identified 37 additional known vulnerabilities beyond the 120 documented in the dataset, and has discovered 51 novel 0-day vulnerabilities in open-source projects.
Paper Structure (25 sections, 6 equations, 4 figures, 3 tables)

This paper contains 25 sections, 6 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background & Related Work
  3. Limitations of Traditional Static Application Security Testing (SAST)
  4. Emerging Applications of LLMs in Security Tasks
  5. LLM-Assisted Static Analysis
  6. Design of vEcho
  7. Key Observation
  8. Stage 1: Candidate Generation & Filtering
  9. Stage 2: Context-Aware Deep Verification
  10. Stage 3: Cognitive Learning & Feedback
  11. Stage 4: Echoic Vulnerability Propagation
  12. Evaluation
  13. Experimental Setup
  14. Research Questions (RQs)
  15. Benchmark Dataset and Evaluation Validity
  16. ...and 10 more sections

Figures (4)

  • Figure 1: Overview of the vEcho framework. The architecture illustrates an intelligent closed loop comprising four stages: (1) Candidate Generation & Filtering, (2) Context-Aware Deep Verification, and (3) Echoic Vulnerability Propagation. This entire process is orchestrated and enhanced by the (4) Cognitive Learning & Feedback module, which feeds learned patterns and new suspicious code back into the verification queue.
  • Figure 2: Breakdown of vEcho's detection rate by CWE type on the CWE-Bench-Java dataset. The percentages and counts (e.g., 83.6% (46/55)) match the data in Section 4.3.
  • Figure 3: Breakdown of the 37 Beyond-the-Benchmark vulnerabilities discovered by vEcho within the CWE-Bench-Java dataset. The chart shows the distribution by CWE type, highlighting a concentration in critical vulnerability classes.
  • Figure 4: Case study of vEcho's 0-day discovery via Echoic Vulnerability Propagation (EVP). (Left) vEcho analyzes a SAST alert in the Resty framework, where a source(snippet 1) is correctly neutralized by a project-specific sanitizer(snippet 3), resulting in a False Positive (FP). (Middle) Instead of discarding the FP, the Verify Agent learns this sanitizer's pattern and generates 'Re-Scan Guidance'. (Right) The EVP mechanism uses this guidance to proactively discover a new, un-sanitized code path in the HttpClient framework, confirming a 0-day Path Traversal vulnerability at the sink(snippet 6).