Cerberus: Real-Time Video Anomaly Detection via Cascaded Vision-Language Models

TL;DR

This work tackles real-time video anomaly detection in the era of vision-language models, where full-scale VLMs are too slow for streaming analytics. Cerberus addresses this by a two-stage cascaded architecture that first filters frames with a lightweight CLIP-based stage and then applies fine-grained VLM reasoning guided by motion masks and offline normality rules. It introduces offline rule induction, motion mask prompting, and rule-based deviation detection to detect unseen anomalies as deviations from learned norms, with user customization and rule evolution for continual improvement. On four datasets, Cerberus delivers real-time throughput (average around 57.68 fps, up to 72.25 fps in some settings) and substantial speedups (around 151.79x) while maintaining state-of-the-art accuracy (≈97.2% AUC), enabling practical edge deployment for safety-critical video analytics.

Abstract

Video anomaly detection (VAD) has rapidly advanced by recent development of Vision-Language Models (VLMs). While these models offer superior zero-shot detection capabilities, their immense computational cost and unstable visual grounding performance hinder real-time deployment. To overcome these challenges, we introduce Cerberus, a two-stage cascaded system designed for efficient yet accurate real-time VAD. Cerberus learns normal behavioral rules offline, and combines lightweight filtering with fine-grained VLM reasoning during online inference. The performance gains of Cerberus come from two key innovations: motion mask prompting and rule-based deviation detection. The former directs the VLM's attention to regions relevant to motion, while the latter identifies anomalies as deviations from learned norms rather than enumerating possible anomalies. Extensive evaluations on four datasets show that Cerberus on average achieves 57.68 fps on an NVIDIA L40S GPU, a 151.79$\times$ speedup, and 97.2\% accuracy comparable to the state-of-the-art VLM-based VAD methods, establishing it as a practical solution for real-time video analytics.

Figures (11)

• Figure 1: Two common VLM-based VAD pipelines. Top: a modular two-step design where a VLM describes video content and an LLM reasons zanella2024lavadye2025verayang2024anomlayruler. Bottom: an integrated single-step design where a full-fledged VLM like Gemini team2023gemini and GPT-4o achiam2023gpt4 handles both perception and reasoning zhao2025smarthomeli2024videovista.
• Figure 2: Architectures of CLIP and a typical VLM.
• Figure 3: An example of attentional distraction: the VLM focuses on the salient foreground objects, thereby missing crucial contextual cues like the traffic sign and the distant violating vehicle.
• Figure 4: Trade-off between anomaly recall and frame filtering proportion under different Top-$k$.
• Figure 5: The system overview of Cerberus.