Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GuardReasoner-Omni: A Reasoning-based Multi-modal Guardrail for Text, Image, and Video

Zhenhao Zhu, Yue Liu, Yanpei Guo, Wenjie Qu, Cancan Chen, Yufei He, Yibo Li, Yulin Chen, Tianyi Wu, Huiying Xu, Xinzhong Zhu, Jiaheng Zhang

TL;DR

GuardReasoner-Omni introduces a unified guardrail capable of reasoning across text, image, and video modalities. It builds GuardReasoner-OmniTrain-148K and employs a two-stage training pipeline: cold-start SFT with explicit reasoning, followed by GRPO with hard-sample mining and an error-driven exploration reward to encourage deeper deliberation. The approach achieves state-of-the-art results across 20 benchmarks, with strong gains in video moderation and notable performance on HarmTextVideo, supported by ablations confirming the benefit of GRPO. It also provides transparent CoT reasoning traces to improve interpretability and trust in omni-modal content moderation, offering practical implications for deploying safer multimodal AI systems.

Abstract

We present GuardReasoner-Omni, a reasoning-based guardrail model designed to moderate text, image, and video data. First, we construct a comprehensive training corpus comprising 148k samples spanning these three modalities. Our training pipeline follows a two-stage paradigm to incentivize the model to deliberate before making decisions: (1) conducting SFT to cold-start the model with explicit reasoning capabilities and structural adherence; and (2) performing RL, incorporating an error-driven exploration reward to incentivize deeper reasoning on hard samples. We release a suite of models scaled at 2B and 4B parameters. Extensive experiments demonstrate that GuardReasoner-Omni achieves superior performance compared to existing state-of-the-art baselines across various guardrail benchmarks. Notably, GuardReasoner-Omni (2B) significantly surpasses the runner-up by 5.3% F1 score.

GuardReasoner-Omni: A Reasoning-based Multi-modal Guardrail for Text, Image, and Video

TL;DR

GuardReasoner-Omni introduces a unified guardrail capable of reasoning across text, image, and video modalities. It builds GuardReasoner-OmniTrain-148K and employs a two-stage training pipeline: cold-start SFT with explicit reasoning, followed by GRPO with hard-sample mining and an error-driven exploration reward to encourage deeper deliberation. The approach achieves state-of-the-art results across 20 benchmarks, with strong gains in video moderation and notable performance on HarmTextVideo, supported by ablations confirming the benefit of GRPO. It also provides transparent CoT reasoning traces to improve interpretability and trust in omni-modal content moderation, offering practical implications for deploying safer multimodal AI systems.

Abstract

We present GuardReasoner-Omni, a reasoning-based guardrail model designed to moderate text, image, and video data. First, we construct a comprehensive training corpus comprising 148k samples spanning these three modalities. Our training pipeline follows a two-stage paradigm to incentivize the model to deliberate before making decisions: (1) conducting SFT to cold-start the model with explicit reasoning capabilities and structural adherence; and (2) performing RL, incorporating an error-driven exploration reward to incentivize deeper reasoning on hard samples. We release a suite of models scaled at 2B and 4B parameters. Extensive experiments demonstrate that GuardReasoner-Omni achieves superior performance compared to existing state-of-the-art baselines across various guardrail benchmarks. Notably, GuardReasoner-Omni (2B) significantly surpasses the runner-up by 5.3% F1 score.
Paper Structure (20 sections, 6 equations, 10 figures, 5 tables)

This paper contains 20 sections, 6 equations, 10 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Dataset Construction
  4. Cold-Start via SFT
  5. Reasoning Enhancement via GRPO
  6. Hard Sample Mining
  7. Reward Design.
  8. Optimization Objective
  9. Experiments
  10. Setup
  11. Main Results
  12. Ablation Study
  13. Case Study
  14. Related Works
  15. Vision-Language Models
  16. ...and 5 more sections

Figures (10)

  • Figure 1: Performance Comparison of Multi-modal Guardrails. The F1 score is calculated by averaging prompt and response harmfulness detection results. Note that for the video modality, we exclude the HarmTextVideo dataset to ensure a fair comparison with Holmes-VAU, which lacks support for text-video inputs.
  • Figure 2: Overview of GuardReasoner-Omni. The framework operates in three stages: (1) Dataset Curation: We curate the GuardReasoner-OmniTrain-148K dataset by distilling CoT reasoning traces from a teacher model and filtering for quality. (2) Cold-Start SFT: The model is fine-tuned to establish explicit reasoning capabilities and strict format compliance. (3) Reasoning Enhancement via GRPO: We first perform hard sample mining to identify inconsistent predictions. Subsequently, we optimize the model using GRPO with a error-driven exploration reward to incentivize deeper reasoning on these challenging cases.
  • Figure 3: Data composition of GuardReasoner-OmniTrain. This dataset comprises 148k samples spanning different modalities. The chart illustrates the proportional distribution of each modality, along with the specific source benchmarks.
  • Figure 4: Case Study on Video Input Data. The model captures the temporal context of the event (a robbery incident) and generates a step-by-step reasoning trace to justify its decision. This highlights the model's capability to handle complex, dynamic safety threats that extend beyond static frame analysis.
  • Figure 5: Prompt template for training and inference.
  • ...and 5 more figures