ProGuard: Towards Proactive Multimodal Safeguard

TL;DR

ProGuard introduces a proactive, reasoning-based guard for multimodal safety that operates without changing base models. By constructing a modality-balanced 87K sample dataset and a hierarchical safety taxonomy, it enables fine-grained, out-of-distribution risk reasoning through online reinforcement learning with specialized rewards, including a synonym-bank-based OOD inference mechanism. Across binary safety, unsafe content categorization, and OOD category inference, ProGuard achieves competitive performance with open-source guards and strong gains in OOD detection and description, supported by human-aligned reward signals. The framework demonstrates the feasibility and value of proactive safety in rapidly evolving multimodal systems and lays groundwork for taxonomy expansion and safer AI deployment in real-world scenarios.

Abstract

The rapid evolution of generative models has led to a continuous emergence of multimodal safety risks, exposing the limitations of existing defense methods. To address these challenges, we propose ProGuard, a vision-language proactive guard that identifies and describes out-of-distribution (OOD) safety risks without the need for model adjustments required by traditional reactive approaches. We first construct a modality-balanced dataset of 87K samples, each annotated with both binary safety labels and risk categories under a hierarchical multimodal safety taxonomy, effectively mitigating modality bias and ensuring consistent moderation across text, image, and text-image inputs. Based on this dataset, we train our vision-language base model purely through reinforcement learning (RL) to achieve efficient and concise reasoning. To approximate proactive safety scenarios in a controlled setting, we further introduce an OOD safety category inference task and augment the RL objective with a synonym-bank-based similarity reward that encourages the model to generate concise descriptions for unseen unsafe categories. Experimental results show that ProGuard achieves performance comparable to closed-source large models on binary safety classification, substantially outperforms existing open-source guard models on unsafe content categorization. Most notably, ProGuard delivers a strong proactive moderation ability, improving OOD risk detection by 52.6% and OOD risk description by 64.8%.

Figures (6)

• Figure 1: Reactive versus proactive moderation pipelines: Reactive guards rely on a fixed safety taxonomy, limiting their ability to identify novel risks. In contrast, proactive guard detects emerging risks and infers appropriate OOD category names. The inferred category shown is a real output generated by ProGuard.
• Figure 2: Overview of ProGuard. The framework combines moderation task design, a multimodal safety taxonomy and a balanced safety dataset, and an online reinforcement learning pipeline to build a reasoning-enhanced proactive guard model.
• Figure 3: Performance comparison of models trained with different methods across five evaluation tasks.
• Figure 4: Human alignment results measured by accuracy.
• Figure 5: Training dynamics under modality imbalance.