Good News for Script Kiddies? Evaluating Large Language Models for Automated Exploit Generation

TL;DR

This work addresses whether modern large language models can be steered to generate automated exploits and how cooperative and effective they are. It introduces a bias-mitigated benchmark built from refactored SEED Labs and an autonomous LLM-based attacker to standardize evaluation across diverse models. The study evaluates five LLMs, including GPT-4/4o, Llama3, and Dolphin variants, and finds that GPT-4/4o are highly cooperative while Llama3 is resistant; none succeed on refactored labs, though GPT-4o achieves only a few errors per attempt, indicating potential for future AEG advances. The contributions include the first systematic AEG evaluation, an LLM-driven attacker framework, and a benchmark with released data and code, providing a foundation for ongoing research in automated exploit generation and security risk assessment.

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in code-related tasks, raising concerns about their potential for automated exploit generation (AEG). This paper presents the first systematic study on LLMs' effectiveness in AEG, evaluating both their cooperativeness and technical proficiency. To mitigate dataset bias, we introduce a benchmark with refactored versions of five software security labs. Additionally, we design an LLM-based attacker to systematically prompt LLMs for exploit generation. Our experiments reveal that GPT-4 and GPT-4o exhibit high cooperativeness, comparable to uncensored models, while Llama3 is the most resistant. However, no model successfully generates exploits for refactored labs, though GPT-4o's minimal errors highlight the potential for LLM-driven AEG advancements.

Figures (3)

• Figure 1: Workflow of the LLM-based attacker
• Figure 2: Number of mistakes made by LLMs on original SEED Lab programs
• Figure 3: Number of mistakes made by LLMs on refactored SEED Lab programs