DETAM: Defending LLMs Against Jailbreak Attacks via Targeted Attention Modification

TL;DR

The paper addresses the vulnerability of LLMs to jailbreak attacks and introduces DETAM, a finetuning-free defense that uses targeted attention modification. By identifying attention heads most sensitive to jailbreaks through differential analysis and redistributing attention to emphasize the user's core intent, DETAM defensively reduces unsafe outputs during inference. It demonstrates strong, model- and attack-agnostic performance with robust generalization, while preserving utility and minimizing false refusals. The approach offers a practical, low-cost defense that can be deployed without retraining and shows promise for broader applicability in safety-critical NLP systems.

Abstract

With the widespread adoption of Large Language Models (LLMs), jailbreak attacks have become an increasingly pressing safety concern. While safety-aligned LLMs can effectively defend against normal harmful queries, they remain vulnerable to such attacks. Existing defense methods primarily rely on fine-tuning or input modification, which often suffer from limited generalization and reduced utility. To address this, we introduce DETAM, a finetuning-free defense approach that improves the defensive capabilities against jailbreak attacks of LLMs via targeted attention modification. Specifically, we analyze the differences in attention scores between successful and unsuccessful defenses to identify the attention heads sensitive to jailbreak attacks. During inference, we reallocate attention to emphasize the user's core intention, minimizing interference from attack tokens. Our experimental results demonstrate that DETAM outperforms various baselines in jailbreak defense and exhibits robust generalization across different attacks and models, maintaining its effectiveness even on in-the-wild jailbreak data. Furthermore, in evaluating the model's utility, we incorporated over-defense datasets, which further validate the superior performance of our approach. The code will be released immediately upon acceptance.

Figures (6)

• Figure 1: An Illustration of LLM Vulnerabilities to Jailbreak Attacks: while safety-aligned LLMs can effectively defend against normal harmful queries, they remain highly susceptible to jailbreak attacks.
• Figure 2: An illustration of DeTAM. (Left) Identifying the attention heads sensitive to jailbreak attacks by analyzing attention distribution differences between successful and unsuccessful defenses. (Right) Dynamically reallocating attention within the identified heads during inference to prioritize the user's intention.
• Figure 3: Ablation study for DeTAM on LLaMA2.
• Figure 4: Impact of Attention Head Selection on DeTAM. (a) Performance comparison of randomly selected vs. sensitive attention heads. (b) Visualization of sensitive attention head selection. Red regions represent attention heads with $\Delta \bar{S}_{i, j}>\alpha$, while blue regions represent attention heads with $\Delta \bar{S}_{i, j}<-\alpha$. (c) $\Delta \bar{S}_{i, j}$ analysis of two groups of failed defense cases, serving as control groups.
• Figure 5: Attention distribution at the last token position to source token regions. We examine the top 5 attention heads most sensitive to jailbreak attacks, comparing their attention distributions under utility and safety prompts.