Exploring Straightforward Conversational Red-Teaming

TL;DR

It is suggested that off-the-shelf models can act as effective red teamers and even adjust their attack strategy based on past attempts, although their effectiveness decreases with greater alignment.

Abstract

Large language models (LLMs) are increasingly used in business dialogue systems but they pose security and ethical risks. Multi-turn conversations, where context influences the model's behavior, can be exploited to produce undesired responses. In this paper, we examine the effectiveness of utilizing off-the-shelf LLMs in straightforward red-teaming approaches, where an attacker LLM aims to elicit undesired output from a target LLM, comparing both single-turn and conversational red-teaming tactics. Our experiments offer insights into various usage strategies that significantly affect their performance as red teamers. They suggest that off-the-shelf models can act as effective red teamers and even adjust their attack strategy based on past attempts, although their effectiveness decreases with greater alignment.

Figures (8)

• Figure 1: An example dialogue between a red-teaming model (red) and the target model (blue) in a conversational setting, with a judge LLM (grey) scoring the harmfulness of the target agent's last responses, but taking the context of the entire conversation into account (harmfulness scores range is [1-5]).
• Figure 2: Average harmfulness scores for each turn of the conversation when Mixtral acts as both the attacker and the target model ($MEAN\pm SEM$).
• Figure 3: Showing the distribution of turns containing the most harmful response, with Mixtral8X7b serving as the attacker and target model (excluding conversations with multiple max scores).
• Figure 4: Distribution of turns containing the most harmful response
• Figure 5: Average harmfulness scores for each turn of the conversation where $\mathcal{A}=\mathcal{T}$.