Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Suffixes: Token Position in GCG Adversarial Attacks on Large Language Models

Hicham Eddoubi, Umar Faruk Abdullahi, Fadi Hassan

TL;DR

This paper examines the safety of large language models against jailbreak prompts, focusing on the Greedy Coordinate Gradient (GCG) attack. It identifies adversarial token position as a new attack axis, showing that optimizing for prefixes and evaluating across both prefix and suffix placements can significantly raise jailbreak success, including in cross-model transfer. The study reveals that attention-based saliency metrics are insufficient for diagnosing vulnerabilities, especially for prefix-based attacks, and highlights that suffix-only evaluations underestimate real-world risk. The findings argue for broader safety evaluations that account for adversarial token position to better assess robustness and deployment risk in LLMs.

Abstract

Large Language Models (LLMs) have seen widespread adoption across multiple domains, creating an urgent need for robust safety alignment mechanisms. However, robustness remains challenging due to jailbreak attacks that bypass alignment via adversarial prompts. In this work, we focus on the prevalent Greedy Coordinate Gradient (GCG) attack and identify a previously underexplored attack axis in jailbreak attacks typically framed as suffix-based: the placement of adversarial tokens within the prompt. Using GCG as a case study, we show that both optimizing attacks to generate prefixes instead of suffixes and varying adversarial token position during evaluation substantially influence attack success rates. Our findings highlight a critical blind spot in current safety evaluations and underline the need to account for the position of adversarial tokens in the adversarial robustness evaluation of LLMs.

Beyond Suffixes: Token Position in GCG Adversarial Attacks on Large Language Models

TL;DR

This paper examines the safety of large language models against jailbreak prompts, focusing on the Greedy Coordinate Gradient (GCG) attack. It identifies adversarial token position as a new attack axis, showing that optimizing for prefixes and evaluating across both prefix and suffix placements can significantly raise jailbreak success, including in cross-model transfer. The study reveals that attention-based saliency metrics are insufficient for diagnosing vulnerabilities, especially for prefix-based attacks, and highlights that suffix-only evaluations underestimate real-world risk. The findings argue for broader safety evaluations that account for adversarial token position to better assess robustness and deployment risk in LLMs.

Abstract

Large Language Models (LLMs) have seen widespread adoption across multiple domains, creating an urgent need for robust safety alignment mechanisms. However, robustness remains challenging due to jailbreak attacks that bypass alignment via adversarial prompts. In this work, we focus on the prevalent Greedy Coordinate Gradient (GCG) attack and identify a previously underexplored attack axis in jailbreak attacks typically framed as suffix-based: the placement of adversarial tokens within the prompt. Using GCG as a case study, we show that both optimizing attacks to generate prefixes instead of suffixes and varying adversarial token position during evaluation substantially influence attack success rates. Our findings highlight a critical blind spot in current safety evaluations and underline the need to account for the position of adversarial tokens in the adversarial robustness evaluation of LLMs.
Paper Structure (13 sections, 1 equation, 10 figures, 2 tables)

This paper contains 13 sections, 1 equation, 10 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Greedy Coordinate Gradient
  4. Experimental method
  5. Experimental setup
  6. Adversarial token position as an attack axis
  7. Attention shift with adversarial token position
  8. Related Works
  9. Conclusions
  10. Additional results on Attention shift
  11. Implementation Details
  12. ASR Evaluation
  13. Examples of GCG failure

Figures (10)

  • Figure 1: Attention scores with GCG-prefix and GCG-suffix attacks for Llama across model layers on successful adversarial prompts. Goal refers to the original harmful prompt
  • Figure 2: Attention scores with GCG-prefix and GCG-suffix attacks for Vicuna across model layers on successful adversarial prompts. Goal refers to the original harmful prompt
  • Figure 3: Attention scores with GCG-prefix and GCG-suffix attacks for Qwen across model layers on successful adversarial prompts. Goal refers to the original harmful prompt
  • Figure 4: Attention scores with GCG-prefix and GCG-suffix attacks for Mistral across model layers on successful adversarial prompts. Goal refers to the original harmful prompt
  • Figure 5: Attention scores with GCG-prefix and GCG-suffix attacks for Deepseek across model layers on successful adversarial prompts. Goal refers to the original harmful prompt
  • ...and 5 more figures