Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM Agents can Autonomously Hack Websites

Richard Fang, Rohan Bindu, Akul Gupta, Qiusi Zhan, Daniel Kang

TL;DR

The paper demonstrates that autonomous LLM agents can hack websites without prior vulnerability knowledge, with GPT-4 achieving up to 73.3% success on a 15-vulnerability sandbox benchmark and even discovering vulnerabilities on real sites. It leverages tool use, planning, and document reading within a headless browser workflow (Playwright) under the OpenAI Assistants API and LangChain, and it shows a clear scaling advantage over open-source models. Ablation studies reveal the critical role of documents and detailed prompts for high performance, and real-world testing indicates non-trivial risk, including potential for responsible disclosure. The work highlights significant security and deployment implications for frontier models and argues for careful release policies and defense-oriented research.

Abstract

In recent years, large language models (LLMs) have become increasingly capable and can now interact with tools (i.e., call functions), read documents, and recursively call themselves. As a result, these LLMs can now function autonomously as agents. With the rise in capabilities of these agents, recent work has speculated on how LLM agents would affect cybersecurity. However, not much is known about the offensive capabilities of LLM agents. In this work, we show that LLM agents can autonomously hack websites, performing tasks as complex as blind database schema extraction and SQL injections without human feedback. Importantly, the agent does not need to know the vulnerability beforehand. This capability is uniquely enabled by frontier models that are highly capable of tool use and leveraging extended context. Namely, we show that GPT-4 is capable of such hacks, but existing open-source models are not. Finally, we show that GPT-4 is capable of autonomously finding vulnerabilities in websites in the wild. Our findings raise questions about the widespread deployment of LLMs.

LLM Agents can Autonomously Hack Websites

TL;DR

The paper demonstrates that autonomous LLM agents can hack websites without prior vulnerability knowledge, with GPT-4 achieving up to 73.3% success on a 15-vulnerability sandbox benchmark and even discovering vulnerabilities on real sites. It leverages tool use, planning, and document reading within a headless browser workflow (Playwright) under the OpenAI Assistants API and LangChain, and it shows a clear scaling advantage over open-source models. Ablation studies reveal the critical role of documents and detailed prompts for high performance, and real-world testing indicates non-trivial risk, including potential for responsible disclosure. The work highlights significant security and deployment implications for frontier models and argues for careful release policies and defense-oriented research.

Abstract

In recent years, large language models (LLMs) have become increasingly capable and can now interact with tools (i.e., call functions), read documents, and recursively call themselves. As a result, these LLMs can now function autonomously as agents. With the rise in capabilities of these agents, recent work has speculated on how LLM agents would affect cybersecurity. However, not much is known about the offensive capabilities of LLM agents. In this work, we show that LLM agents can autonomously hack websites, performing tasks as complex as blind database schema extraction and SQL injections without human feedback. Importantly, the agent does not need to know the vulnerability beforehand. This capability is uniquely enabled by frontier models that are highly capable of tool use and leveraging extended context. Namely, we show that GPT-4 is capable of such hacks, but existing open-source models are not. Finally, we show that GPT-4 is capable of autonomously finding vulnerabilities in websites in the wild. Our findings raise questions about the widespread deployment of LLMs.
Paper Structure (16 sections, 2 figures, 4 tables)

This paper contains 16 sections, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Overview of LLM Agents and Web Security
  3. LLM Agents
  4. Web Security
  5. Leveraging LLM Agents to Hack Websites
  6. LLM Agents can Autonomously Hack Websites
  7. Experimental Setup
  8. Hacking Websites
  9. Ablation Studies
  10. Understanding Agent Capabilities
  11. GPT-4 Case Studies
  12. Open-source LLMs
  13. Hacking Real Websites
  14. Cost Analysis
  15. Related Work
  16. ...and 1 more sections

Figures (2)

  • Figure 1: Schematic of using autonomous LLM agents to hack websites.
  • Figure 2: Ablation experiments with our best performing agent. We removed the detailed prompt, the documents, and both.