RL Is a Hammer and LLMs Are Nails: A Simple Reinforcement Learning Recipe for Strong Prompt Injection

TL;DR

The paper investigates prompt injection risks in LLM agents and evaluates defenses like Instruction Hierarchy and SecAlign, arguing that these defenses are not fundamentally robust. It introduces RL-Hammer, a simple reinforcement-learning recipe that trains attacker models from scratch using Group Relative Policy Optimization to perform strong prompt injections. RL-Hammer achieves very high attack success rates against defended targets (e.g., 98% on GPT-4o and 72% on GPT-5 with Instruction Hierarchy) and demonstrates transferability to unseen models, raising concerns about current defenses. The work also analyzes diversity and detectability, showing that attackers can evade several detectors and that detector-based defenses require more robust strategies; code is provided to enable automatic red-teaming and defense evaluation.

Abstract

Prompt injection poses a serious threat to the reliability and safety of LLM agents. Recent defenses against prompt injection, such as Instruction Hierarchy and SecAlign, have shown notable robustness against static attacks. However, to more thoroughly evaluate the robustness of these defenses, it is arguably necessary to employ strong attacks such as automated red-teaming. To this end, we introduce RL-Hammer, a simple recipe for training attacker models that automatically learn to perform strong prompt injections and jailbreaks via reinforcement learning. RL-Hammer requires no warm-up data and can be trained entirely from scratch. To achieve high ASRs against industrial-level models with defenses, we propose a set of practical techniques that enable highly effective, universal attacks. Using this pipeline, RL-Hammer reaches a 98% ASR against GPT-4o and a $72\%$ ASR against GPT-5 with the Instruction Hierarchy defense. We further discuss the challenge of achieving high diversity in attacks, highlighting how attacker models tend to reward-hack diversity objectives. Finally, we show that RL-Hammer can evade multiple prompt injection detectors. We hope our work advances automatic red-teaming and motivates the development of stronger, more principled defenses. Code is available at https://github.com/facebookresearch/rl-injector.

Figures (3)

• Figure 1: Ablation Study.
• Figure 2: Naive GRPO vs. RL-Hammer.
• Figure 3: Attack Signals on the Robust Model (Meta-SecAlign-8B) During Mid-Training of the Easy Model (Llama-3.1-8B-Instruct).