SafeMT: Multi-turn Safety for Multimodal Language Models

TL;DR

SafeMT introduces a multi-turn, multimodal safety benchmark for evaluating and improving safety in multimodal LLMs. It blends 2,000 harmful queries with images and 8,000 dialogue instances across 17 scenarios, plus four jailbreak strategies, to assess how safety degrades as dialogue length increases. A Safety Index (SI) is defined to capture turn-wise vulnerability and defense consistency, and a Dialogue Safety Moderator is proposed to provide context-specific safety prompts and reduce unsafe outputs; evaluations across 17 models show that longer dialogues and cross-modal inputs raise risk, while the moderator provides meaningful protection. The work highlights critical gaps in current safety mechanisms for multi-turn conversations and sets directions for more robust, domain-aware safety strategies in multimodal dialogue systems, including future work on visual attacks and multi-modal reasoning.

Abstract

With the widespread use of multi-modal Large Language models (MLLMs), safety issues have become a growing concern. Multi-turn dialogues, which are more common in everyday interactions, pose a greater risk than single prompts; however, existing benchmarks do not adequately consider this situation. To encourage the community to focus on the safety issues of these models in multi-turn dialogues, we introduce SafeMT, a benchmark that features dialogues of varying lengths generated from harmful queries accompanied by images. This benchmark consists of 10,000 samples in total, encompassing 17 different scenarios and four jailbreak methods. Additionally, we propose Safety Index (SI) to evaluate the general safety of MLLMs during conversations. We assess the safety of 17 models using this benchmark and discover that the risk of successful attacks on these models increases as the number of turns in harmful dialogues rises. This observation indicates that the safety mechanisms of these models are inadequate for recognizing the hazard in dialogue interactions. We propose a dialogue safety moderator capable of detecting malicious intent concealed within conversations and providing MLLMs with relevant safety policies. Experimental results from several open-source models indicate that this moderator is more effective in reducing multi-turn ASR compared to existed guard models.

Figures (16)

• Figure 1: SafeMT: We integrate existing dataset and manually collect our own red-team queries, incorporating 17 scenarios and four jailbreak methods into our benchmark. The dialogues are generated using closed-source models. Conversations containing refusal words are refined to enhance the potential risk. Dialogue safety moderator is trained based on dialogues, queries and scenarios to reduce the harmfulness of AI generated answers.
• Figure 2: Scenario Distribution
• Figure 3: Dialogue safety moderator is trained on dialogues with images and predicted harmful queries. It will predict the users' intents based on previous conversation and classify the scenario. Security rules will be mapped based on different scenarios and input as system prompt along with users' intent for later inference.
• Figure 4: Comparison of ASR between single prompt and 8-turn dialogues with images in different scenarios. The darker the color, the higher ASR in the scenario.
• Figure 5: Comparison the effectiveness of LLavaGuard and the dialogue safety moderator.