Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattacks

TL;DR

Mantis is a defensive framework that exploits LLMs' susceptibility to adversarial inputs to undermine malicious operations, leading the attacker's LLM to disrupt their own operations or even compromise the attacker's machine.

Abstract

Large language models (LLMs) are increasingly being harnessed to automate cyberattacks, making sophisticated exploits more accessible and scalable. In response, we propose a new defense strategy tailored to counter LLM-driven cyberattacks. We introduce Mantis, a defensive framework that exploits LLMs' susceptibility to adversarial inputs to undermine malicious operations. Upon detecting an automated cyberattack, Mantis plants carefully crafted inputs into system responses, leading the attacker's LLM to disrupt their own operations (passive defense) or even compromise the attacker's machine (active defense). By deploying purposefully vulnerable decoy services to attract the attacker and using dynamic prompt injections for the attacker's LLM, Mantis can autonomously hack back the attacker. In our experiments, Mantis consistently achieved over 95% effectiveness against automated LLM-driven attacks. To foster further research and collaboration, Mantis is available as an open-source tool: https://github.com/pasquini-dario/project_mantis

Figures (11)

• Figure 1: Example of Mantis's defensive prompt injection. In the left panel, a decoy ftp server is spawned by Mantis, which lures the LLM-agent attacker using anonymous credentials. Mantis injects a crafted response into the server's output, tricking the attacker into executing a command that opens a reverse shell on their own machine. In the right panel, Mantis leverages this reverse shell to establish control over the attacker's system.
• Figure 2: Overview of the components of Mantis and its integration within the host system $\mathbf{S}$.
• Figure 3: Example of front login page for Web-app decoy.
• Figure 4: An example of prompt injection hidden using ANSI escape characters to inject a reverse shell into the attacker's machine via a decoy FTP server.
• Figure 5: Example of a hidden banner prompt in the field Server of the Web-app decoy's header.