Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Universal and Black-Box Query-Response Only Attack on LLMs with QROA

Hussein Jawad, Yassine Chenik, Nicolas J. -B. Brunel

TL;DR

QROA introduces a novel black-box jailbreak framework that discovers adversarial suffixes to coerce LLMs into compliance using only the standard query-response interface. Framed as a multi-armed bandit problem, it employs a surrogate neural model and Deep Q-learning with experience replay to efficiently navigate an enormous suffix space without access to logits or internal gradients. A universal extension, QROA-UNV, identifies suffixes that generalize across instructions, enabling one-query jailbreaks on unseen tasks and achieving high ASR across diverse models. The approach demonstrates strong attack efficacy on open- and closed-source LLMs and highlights critical safety considerations and defense needs for robust AI deployment.

Abstract

The rapid adoption of Large Language Models (LLMs) has exposed critical security and ethical vulnerabilities, particularly their susceptibility to adversarial manipulations. This paper introduces QROA, a novel black-box jailbreak method designed to identify adversarial suffixes that can bypass LLM alignment safeguards when appended to a malicious instruction. Unlike existing suffix-based jailbreak approaches, QROA does not require access to the model's logit or any other internal information. It also eliminates reliance on human-crafted templates, operating solely through the standard query-response interface of LLMs. By framing the attack as an optimization bandit problem, QROA employs a surrogate model and token level optimization to efficiently explore suffix variations. Furthermore, we propose QROA-UNV, an extension that identifies universal adversarial suffixes for individual models, enabling one-query jailbreaks across a wide range of instructions. Testing on multiple models demonstrates Attack Success Rate (ASR) greater than 80\%. These findings highlight critical vulnerabilities, emphasize the need for advanced defenses, and contribute to the development of more robust safety evaluations for secure AI deployment. The code is made public on the following link: https://github.com/qroa/QROA

Towards Universal and Black-Box Query-Response Only Attack on LLMs with QROA

TL;DR

QROA introduces a novel black-box jailbreak framework that discovers adversarial suffixes to coerce LLMs into compliance using only the standard query-response interface. Framed as a multi-armed bandit problem, it employs a surrogate neural model and Deep Q-learning with experience replay to efficiently navigate an enormous suffix space without access to logits or internal gradients. A universal extension, QROA-UNV, identifies suffixes that generalize across instructions, enabling one-query jailbreaks on unseen tasks and achieving high ASR across diverse models. The approach demonstrates strong attack efficacy on open- and closed-source LLMs and highlights critical safety considerations and defense needs for robust AI deployment.

Abstract

The rapid adoption of Large Language Models (LLMs) has exposed critical security and ethical vulnerabilities, particularly their susceptibility to adversarial manipulations. This paper introduces QROA, a novel black-box jailbreak method designed to identify adversarial suffixes that can bypass LLM alignment safeguards when appended to a malicious instruction. Unlike existing suffix-based jailbreak approaches, QROA does not require access to the model's logit or any other internal information. It also eliminates reliance on human-crafted templates, operating solely through the standard query-response interface of LLMs. By framing the attack as an optimization bandit problem, QROA employs a surrogate model and token level optimization to efficiently explore suffix variations. Furthermore, we propose QROA-UNV, an extension that identifies universal adversarial suffixes for individual models, enabling one-query jailbreaks across a wide range of instructions. Testing on multiple models demonstrates Attack Success Rate (ASR) greater than 80\%. These findings highlight critical vulnerabilities, emphasize the need for advanced defenses, and contribute to the development of more robust safety evaluations for secure AI deployment. The code is made public on the following link: https://github.com/qroa/QROA
Paper Structure (59 sections, 7 equations, 1 figure, 15 tables, 4 algorithms)

This paper contains 59 sections, 7 equations, 1 figure, 15 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. LLM Jailbreak
  4. Black-Box Jailbreak Attacks on LLMs
  5. Jailbreak through suffix optimization
  6. Query-Response Optimization Attack (QROA)
  7. Formalization
  8. Problem Setting
  9. Objective Function: Single instruction
  10. Objective Function for Universal Suffix Optimization
  11. Optimization Challenges
  12. The Alignment Function
  13. Link with Reinforcement Learning
  14. QROA: A Query Response Optimization Attack
  15. Overview
  16. ...and 44 more sections

Figures (1)

  • Figure 1: Image Illustrating the Methodology of QROA.