One Trigger Token Is Enough: A Defense Strategy for Balancing Safety and Usability in Large Language Models

TL;DR

The paper tackles jailbreak vulnerabilities in safety-aligned LLMs by uncovering shallow safety alignment, where the initial tokens largely determine unsafe outputs. It introduces D-STT, a decoding-based defense that explicitly identifies and decodes safety trigger tokens, constraining the trigger to a single token to preserve usability. Empirical results show safety trigger tokens are surprisingly similar across different harmful inputs, enabling cross-sample generalization and robust defense across multiple models and attacks with minimal latency. The work positions D-STT as a lightweight, deployable defense that can complement deeper safety guardrails, addressing the safety-usability trade-off in practical deployments.

Abstract

Large Language Models (LLMs) have been extensively used across diverse domains, including virtual assistants, automated code generation, and scientific research. However, they remain vulnerable to jailbreak attacks, which manipulate the models into generating harmful responses despite safety alignment. Recent studies have shown that current safety-aligned LLMs undergo shallow safety alignment. In this work, we conduct an in-depth investigation into the underlying mechanism of this phenomenon and reveal that it manifests through learned ''safety trigger tokens'' that activate the model's safety patterns when paired with the specific input. Through both analysis and empirical verification, we further demonstrate the high similarity of the safety trigger tokens across different harmful inputs. Accordingly, we propose D-STT, a simple yet effective defense algorithm that identifies and explicitly decodes safety trigger tokens of the given safety-aligned LLM to activate the model's learned safety patterns. In this process, the safety trigger is constrained to a single token, which effectively preserves model usability by introducing minimum intervention in the decoding process. Extensive experiments across diverse jailbreak attacks and benign prompts demonstrate that D-STT significantly reduces output harmfulness while preserving model usability and incurring negligible response time overhead, outperforming ten baseline methods.

Figures (4)

• Figure 1: The framework of D-STT. We first identify the safety trigger tokens by collecting refusal responses from the given safety-aligned LLM prompted with malicious prompts. Then, we construct a safety-aware prior distribution $P_{\text{safety}}$ by computing the frequency of each distinct safety trigger token appearing in the refusal responses. In the inference phase, we decode the safety trigger tokens by sampling from $P_{\text{safety}}$, and generate the remaining tokens using the normal decoding strategy over the model's conditional distribution.
• Figure 2: Radar chart of average defense rankings over eight jailbreak attacks on four safety-aligned LLMs, based on harmfulness scores.
• Figure 3: System prompts in our experiments.
• Figure 4: Comparison of defense methods in terms of safety and usability on Llama2-7B-chat model. The harmfulness score is calculated as the average across 10 attacks provided by GPT-Judge, while the usability score is computed as the average across 5 dimensions assessed by Just-Eval.