Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SafeVid: Toward Safety Aligned Video Large Multimodal Models

Yixu Wang, Jiaxin Song, Yifeng Gao, Xin Wang, Yang Yao, Yan Teng, Xingjun Ma, Yingchun Wang, Yu-Gang Jiang

TL;DR

The paper addresses safety misalignment in Video Large Multimodal Models (VLMMs) by proposing SafeVid, a framework that transfers robust text-based safety reasoning to video through detailed textual video descriptions. SafeVid comprises three components: SafeVid-350K, a 350K-pair video safety preference dataset; Direct Preference Optimization (DPO) for targeted safety alignment; and SafeVidBench, a comprehensive video-focused safety benchmark. Empirical results show substantial safety improvements on SafeVidBench and favorable generalization to OOD safety benchmarks with a modest impact on general video capabilities, supported by scalable data experiments. The work provides public resources and demonstrates that textual grounding can effectively instill video-specific safety principles in VLMMs, with practical implications for safer deployment.

Abstract

As Video Large Multimodal Models (VLMMs) rapidly advance, their inherent complexity introduces significant safety challenges, particularly the issue of mismatched generalization where static safety alignments fail to transfer to dynamic video contexts. We introduce SafeVid, a framework designed to instill video-specific safety principles in VLMMs. SafeVid uniquely transfers robust textual safety alignment capabilities to the video domain by employing detailed textual video descriptions as an interpretive bridge, facilitating LLM-based rule-driven safety reasoning. This is achieved through a closed-loop system comprising: 1) generation of SafeVid-350K, a novel 350,000-pair video-specific safety preference dataset; 2) targeted alignment of VLMMs using Direct Preference Optimization (DPO); and 3) comprehensive evaluation via our new SafeVidBench benchmark. Alignment with SafeVid-350K significantly enhances VLMM safety, with models like LLaVA-NeXT-Video demonstrating substantial improvements (e.g., up to 42.39%) on SafeVidBench. SafeVid provides critical resources and a structured approach, demonstrating that leveraging textual descriptions as a conduit for safety reasoning markedly improves the safety alignment of VLMMs. We have made SafeVid-350K dataset (https://huggingface.co/datasets/yxwang/SafeVid-350K) publicly available.

SafeVid: Toward Safety Aligned Video Large Multimodal Models

TL;DR

The paper addresses safety misalignment in Video Large Multimodal Models (VLMMs) by proposing SafeVid, a framework that transfers robust text-based safety reasoning to video through detailed textual video descriptions. SafeVid comprises three components: SafeVid-350K, a 350K-pair video safety preference dataset; Direct Preference Optimization (DPO) for targeted safety alignment; and SafeVidBench, a comprehensive video-focused safety benchmark. Empirical results show substantial safety improvements on SafeVidBench and favorable generalization to OOD safety benchmarks with a modest impact on general video capabilities, supported by scalable data experiments. The work provides public resources and demonstrates that textual grounding can effectively instill video-specific safety principles in VLMMs, with practical implications for safer deployment.

Abstract

As Video Large Multimodal Models (VLMMs) rapidly advance, their inherent complexity introduces significant safety challenges, particularly the issue of mismatched generalization where static safety alignments fail to transfer to dynamic video contexts. We introduce SafeVid, a framework designed to instill video-specific safety principles in VLMMs. SafeVid uniquely transfers robust textual safety alignment capabilities to the video domain by employing detailed textual video descriptions as an interpretive bridge, facilitating LLM-based rule-driven safety reasoning. This is achieved through a closed-loop system comprising: 1) generation of SafeVid-350K, a novel 350,000-pair video-specific safety preference dataset; 2) targeted alignment of VLMMs using Direct Preference Optimization (DPO); and 3) comprehensive evaluation via our new SafeVidBench benchmark. Alignment with SafeVid-350K significantly enhances VLMM safety, with models like LLaVA-NeXT-Video demonstrating substantial improvements (e.g., up to 42.39%) on SafeVidBench. SafeVid provides critical resources and a structured approach, demonstrating that leveraging textual descriptions as a conduit for safety reasoning markedly improves the safety alignment of VLMMs. We have made SafeVid-350K dataset (https://huggingface.co/datasets/yxwang/SafeVid-350K) publicly available.
Paper Structure (10 sections, 1 equation, 4 figures, 3 tables)

This paper contains 10 sections, 1 equation, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. SafeVid
  4. SafeVid-350K: Safety Alignment Dataset Construction
  5. Direct Preference Optimization Based Safety Alignment
  6. SafeVidBench: Comprehensive Safety Evaluation
  7. Experiments
  8. Experimental Setup
  9. Experimental Results
  10. Conclusion

Figures (4)

  • Figure 1: Illustration of mismatched generalization in VLMMs (left) and the improvement of SafeVid (right). Left: Given a harmful query and relevant video context, the unaligned LLaVA-NeXT-Video generates dangerous instructions. After alignment on our SafeVid-350K dataset, the model safely refuses and provides ethical guidance. Right: The safety improvement (outer bars) of LLaVA-Next-Video on SafeVidBench after fine-tuning on our SafeVid-350K dataset.
  • Figure 2: Overview of the SafeVid-350K construction framework. Left: The hierarchical scene classification system detailing the distribution of 12,377 curated videos across 30 distinct scene categories. Right: The structured safety dimensions, organized under the Helpful, Honest, and Harmless (3H) principles and their subcategories, which guide the adversarial question generation and preference pair synthesis for the dataset.
  • Figure 3: Impact of SafeVid-350K data scale on alignment effectiveness. Safety Rate is evaluated on SafeVidBench and MM-SafetyBench.
  • Figure 4: Impact of SafeVid alignment on general VLMM capabilities, evaluated on MMBench-Video. Performance scores (higher is better) on perception (CP: Coarse Perception, FP-S: Fine-grained Perception [Single-Instance], FP-C: Fine-grained Perception [Cross-Instance], HL: Hallucination) and reasoning (LR: Logic Reasoning, AR: Attribute Reasoning, RR: Relation Reasoning, CSR: Common Sense Reasoning, TR: Temporal Reasoning) categories are shown for LLaVA-NeXT-Video and Qwen2.5-VL before and after alignment with SafeVid-350K.