Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Red-Bandit: Test-Time Adaptation for LLM Red-Teaming via Bandit-Guided LoRA Experts

Christos Ziakas, Nicholas Loo, Nishita Jain, Alessandra Russo

TL;DR

Red-Bandit tackles test-time adaptation for automated LLM red-teaming by learning a suite of style-conditioned LoRA experts trained with GRPO and regulated by a rule-based safety reward. At inference, a bandit policy (e.g., UCB or ε-greedy) dynamically selects among attack styles to maximize unsafe outputs from the target model, balancing exploration of stylistic diversity with exploitation of effective vulnerabilities. The approach delivers state-of-the-art ASR@10 on AdvBench across open-source and proprietary LLMs, while producing more fluent prompts, and furthermore provides a diagnostic view into model-specific weaknesses by analyzing attack-style distributions. Overall, Red-Bandit advances automated red-teaming by enabling efficient, test-time adaptation and interpretable vulnerability profiling, with practical implications for pre-deployment safety auditing and model alignment.

Abstract

Automated red-teaming has emerged as a scalable approach for auditing Large Language Models (LLMs) prior to deployment, yet existing approaches lack mechanisms to efficiently adapt to model-specific vulnerabilities at inference. We introduce Red-Bandit, a red-teaming framework that adapts online to identify and exploit model failure modes under distinct attack styles (e.g., manipulation, slang). Red-Bandit post-trains a set of parameter-efficient LoRA experts, each specialized for a particular attack style, using reinforcement learning that rewards the generation of unsafe prompts via a rule-based safety model. At inference, a multi-armed bandit policy dynamically selects among these attack-style experts based on the target model's response safety, balancing exploration and exploitation. Red-Bandit achieves state-of-the-art results on AdvBench under sufficient exploration (ASR@10), while producing more human-readable prompts (lower perplexity). Moreover, Red-Bandit's bandit policy serves as a diagnostic tool for uncovering model-specific vulnerabilities by indicating which attack styles most effectively elicit unsafe behaviors.

Red-Bandit: Test-Time Adaptation for LLM Red-Teaming via Bandit-Guided LoRA Experts

TL;DR

Red-Bandit tackles test-time adaptation for automated LLM red-teaming by learning a suite of style-conditioned LoRA experts trained with GRPO and regulated by a rule-based safety reward. At inference, a bandit policy (e.g., UCB or ε-greedy) dynamically selects among attack styles to maximize unsafe outputs from the target model, balancing exploration of stylistic diversity with exploitation of effective vulnerabilities. The approach delivers state-of-the-art ASR@10 on AdvBench across open-source and proprietary LLMs, while producing more fluent prompts, and furthermore provides a diagnostic view into model-specific weaknesses by analyzing attack-style distributions. Overall, Red-Bandit advances automated red-teaming by enabling efficient, test-time adaptation and interpretable vulnerability profiling, with practical implications for pre-deployment safety auditing and model alignment.

Abstract

Automated red-teaming has emerged as a scalable approach for auditing Large Language Models (LLMs) prior to deployment, yet existing approaches lack mechanisms to efficiently adapt to model-specific vulnerabilities at inference. We introduce Red-Bandit, a red-teaming framework that adapts online to identify and exploit model failure modes under distinct attack styles (e.g., manipulation, slang). Red-Bandit post-trains a set of parameter-efficient LoRA experts, each specialized for a particular attack style, using reinforcement learning that rewards the generation of unsafe prompts via a rule-based safety model. At inference, a multi-armed bandit policy dynamically selects among these attack-style experts based on the target model's response safety, balancing exploration and exploitation. Red-Bandit achieves state-of-the-art results on AdvBench under sufficient exploration (ASR@10), while producing more human-readable prompts (lower perplexity). Moreover, Red-Bandit's bandit policy serves as a diagnostic tool for uncovering model-specific vulnerabilities by indicating which attack styles most effectively elicit unsafe behaviors.

Paper Structure

This paper contains 37 sections, 4 equations, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. RL Post-Training with GRPO
  4. Multi-Armed Bandits
  5. Methodology
  6. Automated Red-teaming
  7. Post-Training of LoRA Experts
  8. RL Algorithm
  9. Reward Function Design
  10. Attack-Style LoRA Experts
  11. Bandit-Guided Inference
  12. Experiments
  13. Experiment Setup
  14. Implementation Details
  15. Evaluation Setup
  16. ...and 22 more sections

Figures (2)

  • Figure 1: Inference pipeline of Red-Bandit. At test time, a multi-armed bandit dynamically selects among attack-style LoRA experts based on response-safety reward from the target model, balancing exploration (stylistic diversity) and exploitation (attack success).
  • Figure 2: Training pipeline of Red-Bandit. Multiple LoRA attack-style experts are trained with GRPO using rule-based rewards on prompt safety.