Beyond Fixed and Dynamic Prompts: Embedded Jailbreak Templates for Advancing LLM Security

TL;DR

The paper addresses the risk of jailbreak prompts in LLMs and the shortcomings of fixed and dynamic templates. It introduces Embedded Jailbreak Template (EJT), which embeds harmful queries within an existing template while preserving its structure, aided by progressive prompt engineering and standardized evaluation. Through extensive intrinsic and comparative analyses, EJT delivers richer embedding-space diversity, higher intent preservation (86.59%), and stronger attack effectiveness (ASR 2.40) than existing templates, highlighting its practicality for red-teaming and policy regression testing. Overall, EJT offers a realistic, reproducible benchmarking framework that advances LLM safety evaluation and defense design by balancing fidelity, diversity, and automation.

Abstract

As the use of large language models (LLMs) continues to expand, ensuring their safety and robustness has become a critical challenge. In particular, jailbreak attacks that bypass built-in safety mechanisms are increasingly recognized as a tangible threat across industries, driving the need for diverse templates to support red-teaming efforts and strengthen defensive techniques. However, current approaches predominantly rely on two limited strategies: (i) substituting harmful queries into fixed templates, and (ii) having the LLM generate entire templates, which often compromises intent clarity and reproductibility. To address this gap, this paper introduces the Embedded Jailbreak Template, which preserves the structure of existing templates while naturally embedding harmful queries within their context. We further propose a progressive prompt-engineering methodology to ensure template quality and consistency, alongside standardized protocols for generation and evaluation. Together, these contributions provide a benchmark that more accurately reflects real-world usage scenarios and harmful intent, facilitating its application in red-teaming and policy regression testing.

Figures (7)

• Figure 1: Fixed Jailbreak Template Example
• Figure 2: Embedded Jailbreak Template Example
• Figure 3: Progressive Prompt Engineering
• Figure 4: Template similarity and refusal across progressive prompt engineering
• Figure 5: Embedding space visualization using PCA (a) and UMAP (b)