Video-SafetyBench: A Benchmark for Safety Evaluation of Video LVLMs

TL;DR

Video-SafetyBench introduces the first benchmark for evaluating safety of video text-enabled systems under video-text attacks, featuring 2,264 video-text pairs across 13 unsafe categories and 48 subcategories built from 1,132 synthesized 10-second videos. It uses a controllable pipeline that splits video semantics into subject images and motion text to generate query-relevant videos, and introduces RJScore, an LLM-based, confidence-aware safety metric calibrated to human judgments. Large-scale experiments across 24 LVLMs reveal that video inputs increase safety risk relative to static images, with benign queries often triggering unsafe outputs and larger models not necessarily offering better safety. The work provides a practical dataset, generation pipeline, and evaluation toolset to drive future defense research in temporal multimodal safety, highlighting gaps in current safety alignments and prompting development of temporal-aware defenses.

Abstract

The increasing deployment of Large Vision-Language Models (LVLMs) raises safety concerns under potential malicious inputs. However, existing multimodal safety evaluations primarily focus on model vulnerabilities exposed by static image inputs, ignoring the temporal dynamics of video that may induce distinct safety risks. To bridge this gap, we introduce Video-SafetyBench, the first comprehensive benchmark designed to evaluate the safety of LVLMs under video-text attacks. It comprises 2,264 video-text pairs spanning 48 fine-grained unsafe categories, each pairing a synthesized video with either a harmful query, which contains explicit malice, or a benign query, which appears harmless but triggers harmful behavior when interpreted alongside the video. To generate semantically accurate videos for safety evaluation, we design a controllable pipeline that decomposes video semantics into subject images (what is shown) and motion text (how it moves), which jointly guide the synthesis of query-relevant videos. To effectively evaluate uncertain or borderline harmful outputs, we propose RJScore, a novel LLM-based metric that incorporates the confidence of judge models and human-aligned decision threshold calibration. Extensive experiments show that benign-query video composition achieves average attack success rates of 67.2%, revealing consistent vulnerabilities to video-induced attacks. We believe Video-SafetyBench will catalyze future research into video-based safety evaluation and defense strategies.

Figures (32)

• Figure 1: Two primary video-text attack compositions to induce unsafe outputs in LVLMs.
• Figure 2: Overview of safety risk taxonomy in Video-SafetyBench. The dataset includes 13 unsafe categories and 48 subcategories, with each video paired with both harmful (H) and benign (B) queries.
• Figure 3: Overview of the Video-SafetyBench construction pipeline. The pipeline involves a three-stage process: Stage 1 (Text): Generation of harmful and benign textual queries based on predefined safety policies. Stage 2 (Text → Image): Generation of subject images via LLM-guided prompts enriched with concrete descriptions. Stage 3 (Image + Text → Video): Generation of query-relevant videos conditioned on both the subject image and LVLM-driven motion trajectories.
• Figure 4: Comparison of judge models in aligning with human evaluation across 1,132 query-response pairs. We use the majority vote of three expert annotators as the ground truth label.
• Figure 5: Five-fold cross-validation is used to select the optimal threshold that best aligns RJScore with human safety annotations.