A Prompt-Based Framework for Loop Vulnerability Detection Using Local LLMs
Adeyemi Adeseye, Aisvarya Adeseye
TL;DR
The paper addresses the challenge of detecting loop-related vulnerabilities that elude traditional static analyzers by leveraging a prompt-based framework for local LLMs to analyze Python code. It introduces a three-process methodology—manual baseline creation, automated vulnerability extraction with two small local models (LLaMA 3.2 and Phi 3.5) guided by iterative prompts, and validation against a ground truth—evaluating three vulnerability categories: control/logic errors, security risks, and resource inefficiencies. Results show that Phi outperforms LLaMA in precision, recall, and F1 across categories, achieving about $F1 \approx 0.90$ for most categories and $F1 \approx 0.95$ for resource management, demonstrating the effectiveness of structured, code-aware prompting for offline vulnerability analysis. The work highlights the privacy and latency benefits of local LLMs and points to future work on concurrency detection, multi-language support, and IDE integration to broaden practical impact.
Abstract
Loop vulnerabilities are one major risky construct in software development. They can easily lead to infinite loops or executions, exhaust resources, or introduce logical errors that degrade performance and compromise security. The problem are often undetected by traditional static analyzers because such tools rely on syntactic patterns, which makes them struggle to detect semantic flaws. Consequently, Large Language Models (LLMs) offer new potential for vulnerability detection because of their ability to understand code contextually. Moreover, local LLMs unlike commercial ones like ChatGPT or Gemini addresses issues such as privacy, latency, and dependency concerns by facilitating efficient offline analysis. Consequently, this study proposes a prompt-based framework that utilize local LLMs for the detection of loop vulnerabilities within Python 3.7+ code. The framework targets three categories of loop-related issues, such as control and logic errors, security risks inside loops, and resource management inefficiencies. A generalized and structured prompt-based framework was designed and tested with two locally deployed LLMs (LLaMA 3.2; 3B and Phi 3.5; 4B) by guiding their behavior via iterative prompting. The designed prompt-based framework included key safeguarding features such as language-specific awareness, code-aware grounding, version sensitivity, and hallucination prevention. The LLM results were validated against a manually established baseline truth, and the results indicate that Phi outperforms LLaMA in precision, recall, and F1-score. The findings emphasize the importance of designing effective prompts for local LLMs to perform secure and accurate code vulnerability analysis.