Leveraging Large Language Models for Command Injection Vulnerability Analysis in Python: An Empirical Study on Popular Open-Source Projects

TL;DR

This paper investigates the use of large language models (LLMs) to detect command injection vulnerabilities and generate security tests in Python across six popular open-source projects. By analyzing 13,037 Python files and 190 candidate functions, the study benchmarks GPT-4, GPT-4o, Claude 3.5 Sonnet, and DeepSeek-R1 against Bandit, revealing that GPT-4 achieves the highest vulnerability-detection accuracy (75.5%) and a strong recall (81.7%), while DeepSeek-R1 excels at test generation. The work highlights both strengths and limitations of LLM-based analysis, notably misses on certain subprocess patterns and the practicalities of runnable test generation, suggesting a complementary role for LLMs alongside traditional tools. A publicly released dataset from six projects provides a benchmark for future research, and the findings motivate hybrid approaches that combine LLMs with static analysis to improve coverage and reliability in software security testing.

Abstract

Command injection vulnerabilities are a significant security threat in dynamic languages like Python, particularly in widely used open-source projects where security issues can have extensive impact. With the proven effectiveness of Large Language Models(LLMs) in code-related tasks, such as testing, researchers have explored their potential for vulnerabilities analysis. This study evaluates the potential of large language models (LLMs), such as GPT-4, as an alternative approach for automated testing for vulnerability detection. In particular, LLMs have demonstrated advanced contextual understanding and adaptability, making them promising candidates for identifying nuanced security vulnerabilities within code. To evaluate this potential, we applied LLM-based analysis to six high-profile GitHub projects-Django, Flask, TensorFlow, Scikit-learn, PyTorch, and Langchain-each with over 50,000 stars and extensive adoption across software development and academic research. Our analysis assesses both the strengths and limitations of LLMs in detecting command injection vulnerabilities, evaluating factors such as detection accuracy, efficiency, and practical integration into development workflows. In addition, we provide a comparative analysis of different LLM tools to identify those most suitable for security applications. Our findings offer guidance for developers and security researchers on leveraging LLMs as innovative and automated approaches to enhance software security.

Figures (8)

• Figure 1: The detection results for GPT-4 and Bandit.
• Figure 2: The overview of the proposed LLM-based approach.
• Figure 3: LLM prompt for vulnerability analysis and security testing generation.
• Figure 4: Example workflow of the proposed LLM-based approach.
• Figure 5: The comparison results of the four LLMs in command injection vulnerabilities detection performance.