VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding

TL;DR

VideoHallu tackles whether vision–language models truly reason about visual content or rely on language priors. It introduces a synthetic, counterfactual video benchmark with expert-annotated QA spanning alignment, spatial–temporal consistency, commonsense, and physics, and evaluates 17 SOTA VLMs using an LLM-based judge. The results reveal widespread hallucinations and limited reasoning on synthetic abnormalities, especially in physics and commonsense tasks. Training with a mix of synthetic and real data via GRPO improves performance on VideoHallu while preserving real-world benchmark results, underscoring the value of reasoning-focused data for robust multimodal understanding.

Abstract

Vision-Language Models (VLMs) have achieved strong results in video understanding, yet a key question remains: do they truly comprehend visual content or only learn shallow correlations between vision and language? Real visual understanding, especially of physics and common sense, is essential for AI systems that interact with the physical world. Current evaluations mostly use real-world videos similar to training data, so high benchmark scores may not reflect real reasoning ability. To address this, we propose negative-control tests using videos that depict physically impossible or logically inconsistent events. We introduce VideoHallu, a synthetic dataset of physics- and commonsense-violating scenes generated with Veo2, Sora, and Kling. It includes expert-annotated question-answer pairs across four categories of violations. Tests of leading VLMs (Qwen-2.5-VL, Video-R1, VideoChat-R1) show that, despite strong results on benchmarks such as MVBench and MMVU, they often miss these violations, exposing gaps in visual reasoning. Reinforcement learning fine-tuning on VideoHallu improves recognition of such violations without reducing standard benchmark performance. Our data is available at https://github.com/zli12321/VideoHallu.git.

Figures (18)

• Figure 1: Illustrative examples of designed negative-control tests to evaluate the critical thinking abilities of VLMs. Unlike real-world videos, synthetic videos can contain counterfactual or commonsense-violating contexts misaligned with reality. VideoHallu includes such synthetic videos with perceptually obvious abnormalities, paired with crafted questions that probe counterintuitive phenomena or test VLMs’ critical thinking in detecting such abnormalities. When SOTA VLMs are evaluated on VideoHallu, they frequently hallucinate, which suggests that these models rely on language priors and commonsense knowledge rather than truly understand the videos.
• Figure 2: Question Categorization of VideoHallu. We design our benchmark, VideoHallu, with four question categories to probe limitations in synthetic video understanding, covering perceptual understanding to abstract reasoning: (a) Physics assesses if the model applies physical laws to entity motions and procedural understanding. (b) Common Sense Reasoning tests if the model can reason based on its knowledge. (c) Spatial-temporal Consistency examines whether the model can track entity motion across frames. (d) Alignment checks if the model correctly identifies and understands entities using visual and textual cues.
• Figure 3: Example Synthetic Videos in VideoHallu. Example hallucination cases observed during SOTA VLM evaluations on synthetic video tasks. Each example includes the generation prompt, key frames, questions, human-annotated ground truth, and hallucinated answers from GPT-4o, Qwen2.5-VL, and Gemini-2.5-Pro, with hallucinations marked in Red.
• Figure 4: Hallucination Case from Alignment – Entity Counting (A-EC). We show hallucination examples from SOTA MLLM evaluations under the A-EC category. Each case includes the video generation prompt (Gray), key frames from synthetic videos (Gray), questions (Orange), ground truth (Green), and model answers from GPT-4o (Black), Qwen2.5-VL (Purple), and Gemini-2.5-Pro (Blue), with hallucinations and critical context highlighted in Red.
• Figure 5: Hallucination Case from Alignment – Entity Properties (A-EP). We show hallucination examples from SOTA MLLM evaluations under the A-EP category. Each case includes the video generation prompt (Gray), key frames from synthetic videos (Gray), questions (Orange), ground truth (Green), and model answers from GPT-4o (Black), Qwen2.5-VL (Purple), and Gemini-2.5-Pro (Blue), with hallucinations and critical context highlighted in Red.