Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FERRET: Framework for Expansion Reliant Red Teaming

Ninareh Mehrabi, Vitor Albiero, Maya Pavlova, Joanna Bitton

Abstract

We introduce a multi-faceted automated red teaming framework in which the goal is to generate multi-modal adversarial conversations that would break a target model and introduce various expansions that would result in more effective and efficient adversarial conversations. The introduced expansions include: 1. Horizontal expansion in which the goal is for the red team model to self-improve and generate more effective conversation starters that would shape a conversation. 2. Vertical expansion in which the goal is to take these conversation starters that are discovered in the horizontal expansion phase and expand them into effective multi-modal conversations and 3. Meta expansion in which the goal is for the red team model to discover more effective multi-modal attack strategies during the course of a conversation. We call our framework FERRET (Framework for Expansion Reliant Red Teaming) and compare it with various existing automated red teaming approaches. In our experiments, we demonstrate the effectiveness of FERRET in generating effective multi-modal adversarial conversations and its superior performance against existing state of the art approaches.

FERRET: Framework for Expansion Reliant Red Teaming

Abstract

We introduce a multi-faceted automated red teaming framework in which the goal is to generate multi-modal adversarial conversations that would break a target model and introduce various expansions that would result in more effective and efficient adversarial conversations. The introduced expansions include: 1. Horizontal expansion in which the goal is for the red team model to self-improve and generate more effective conversation starters that would shape a conversation. 2. Vertical expansion in which the goal is to take these conversation starters that are discovered in the horizontal expansion phase and expand them into effective multi-modal conversations and 3. Meta expansion in which the goal is for the red team model to discover more effective multi-modal attack strategies during the course of a conversation. We call our framework FERRET (Framework for Expansion Reliant Red Teaming) and compare it with various existing automated red teaming approaches. In our experiments, we demonstrate the effectiveness of FERRET in generating effective multi-modal adversarial conversations and its superior performance against existing state of the art approaches.
Paper Structure (18 sections, 3 figures, 4 tables)

This paper contains 18 sections, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. FERRET
  3. Framework
  4. Horizontal Expansion
  5. Vertical Expansion
  6. Meta Expansion
  7. Experiments and Results
  8. Main Experiments
  9. Main Results
  10. Single Turn Ablations
  11. Single Turn Ablation Results
  12. Sampling Ablations
  13. Sampling Ablations Results
  14. Human Studies
  15. Human Studies Results
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Overview of the FERRET framework. The framework gets policy descriptions, attack strategies, and few-shot examples as inputs. In horizontal expansion, the attack model self-evolves and explores what conversation starters are more effective by logging them in the horizontal feedback logs. The attacks are turned into multi-modal attacks using the transformation toolkit that takes the attack in XML format and applies appropriate transformations depending on the attack to create a multi-modal version of the attack. In vertical expansion, the conversation starters are expanded into full conversations. Once the conversation is fully formed, the conversation is saved in conversation logs and next conversation starter is generated.
  • Figure 2: TSNE plots demonstrating diversity results for the main experiments comparing different approaches with regards to the diversity of the generated attacks for each policy.
  • Figure 3: TSNE plots demonstrating diversity results for the ablation study comparing FERRET to FLIRT in single-turn setup with regards to the diversity of the generated attacks for each policy.