Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree

Wenlong Li, Yifei Xu, Yuan Rao, Zhenhua Wang, Shuiguang Deng

TL;DR

VADTree tackles the challenge of training-free video anomaly detection by introducing a Hierarchical Granularity-aware Tree (HGTree) that adaptively samples video content across coarse and fine temporal scales using pre-trained GEBD boundaries. It fuses multidimensional priors from LLMs with VLM content descriptions, refines node scores within intra-cluster neighborhoods, and resolves cross-scale conflicts via inter-cluster cohesion to produce robust anomaly scores without domain-specific training. The method achieves state-of-the-art results among training-free, unsupervised, and some weakly supervised baselines across UCF-Crime, XD-Violence, and MSAD, while reducing the number of sampled segments and providing interpretable, node-wise explanations. This approach offers practical benefits for real-world surveillance by combining explainable reasoning with flexible, data-efficient anomaly detection, albeit relying on the capabilities and limitations of VLM/LLM systems and raising privacy considerations that warrant safeguards.

Abstract

Video anomaly detection (VAD) focuses on identifying anomalies in videos. Supervised methods demand substantial in-domain training data and fail to deliver clear explanations for anomalies. In contrast, training-free methods leverage the knowledge reserves and language interactivity of large pre-trained models to detect anomalies. However, the current fixed-length temporal window sampling approaches struggle to accurately capture anomalies with varying temporal spans. Therefore, we propose VADTree that utilizes a Hierarchical Granularityaware Tree (HGTree) structure for flexible sampling in VAD. VADTree leverages the knowledge embedded in a pre-trained Generic Event Boundary Detection (GEBD) model to characterize potential anomaly event boundaries. Specifically, VADTree decomposes the video into generic event nodes based on boundary confidence, and performs adaptive coarse-fine hierarchical structuring and redundancy removal to construct the HGTree. Then, the multi-dimensional priors are injected into the visual language models (VLMs) to enhance the node-wise anomaly perception, and anomaly reasoning for generic event nodes is achieved via large language models (LLMs). Finally, an inter-cluster node correlation method is used to integrate the multi-granularity anomaly scores. Extensive experiments on three challenging datasets demonstrate that VADTree achieves state-of-the-art performance in training-free settings while drastically reducing the number of sampled video segments. The code will be available at https://github.com/wenlongli10/VADTree.

VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree

TL;DR

VADTree tackles the challenge of training-free video anomaly detection by introducing a Hierarchical Granularity-aware Tree (HGTree) that adaptively samples video content across coarse and fine temporal scales using pre-trained GEBD boundaries. It fuses multidimensional priors from LLMs with VLM content descriptions, refines node scores within intra-cluster neighborhoods, and resolves cross-scale conflicts via inter-cluster cohesion to produce robust anomaly scores without domain-specific training. The method achieves state-of-the-art results among training-free, unsupervised, and some weakly supervised baselines across UCF-Crime, XD-Violence, and MSAD, while reducing the number of sampled segments and providing interpretable, node-wise explanations. This approach offers practical benefits for real-world surveillance by combining explainable reasoning with flexible, data-efficient anomaly detection, albeit relying on the capabilities and limitations of VLM/LLM systems and raising privacy considerations that warrant safeguards.

Abstract

Video anomaly detection (VAD) focuses on identifying anomalies in videos. Supervised methods demand substantial in-domain training data and fail to deliver clear explanations for anomalies. In contrast, training-free methods leverage the knowledge reserves and language interactivity of large pre-trained models to detect anomalies. However, the current fixed-length temporal window sampling approaches struggle to accurately capture anomalies with varying temporal spans. Therefore, we propose VADTree that utilizes a Hierarchical Granularityaware Tree (HGTree) structure for flexible sampling in VAD. VADTree leverages the knowledge embedded in a pre-trained Generic Event Boundary Detection (GEBD) model to characterize potential anomaly event boundaries. Specifically, VADTree decomposes the video into generic event nodes based on boundary confidence, and performs adaptive coarse-fine hierarchical structuring and redundancy removal to construct the HGTree. Then, the multi-dimensional priors are injected into the visual language models (VLMs) to enhance the node-wise anomaly perception, and anomaly reasoning for generic event nodes is achieved via large language models (LLMs). Finally, an inter-cluster node correlation method is used to integrate the multi-granularity anomaly scores. Extensive experiments on three challenging datasets demonstrate that VADTree achieves state-of-the-art performance in training-free settings while drastically reducing the number of sampled video segments. The code will be available at https://github.com/wenlongli10/VADTree.
Paper Structure (52 sections, 1 theorem, 16 equations, 7 figures, 11 tables, 1 algorithm)

This paper contains 52 sections, 1 theorem, 16 equations, 7 figures, 11 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Video Anomaly Detection
  4. Event-based Video Understanding
  5. Methodology
  6. Hierarchical Granularity-aware Tree
  7. Segmentation Confidence Sequence
  8. Generic Event Node Initialization
  9. Adaptive Node Stratification
  10. Generic Event-centric Anomaly Scoring and Refining
  11. Prior-infused Node Scoring
  12. Intra-cluster Node Refinement
  13. Inter-cluster Node Correlation
  14. Experiments
  15. Datasets
  16. ...and 37 more sections

Key Result

Theorem 1

Based on the method described in Section sec:3.1.3, we get $\mathcal{T}' = (\mathcal{S}_{coarse}', \mathcal{S}_{fine}')$, where $|\mathcal{S}_{coarse}'| = M_c'$ and $|\mathcal{S}_{fine}'| = M_f'$ . Then: The original video sequence $V_{1:T}$ can be exactly reconstructed through temporal concatena

Figures (7)

  • Figure 1: Comparison of our methods with popular paradigms. As illustrated in (A), prevailing training-free VAD methods relying on fixed-length sliding temporal window sampling inherently fail to adapt to dynamic anomaly durations. (B) demonstrates our VADTree is based on pre-trained knowledge of Generic Event Boundary Detection to achieve adaptive coarse-fine hierarchical representation of videos, and support multi-granularity anomaly understanding and score fusion. (C) displays the maximum IoU between all sampled video segments and ground-truth abnormal events across two VAD datasets. The sampling results of 10 seconds long fixed-length sliding temporal window (TW) ye2024verazanella2024harnessing can only achieve higher IoU with abnormal events that are close in length to itself ($\text{mIoU}=0.51$ on UCF-Crime and $\text{mIoU}=0.44$ on XD-Violence). Our granularity-aware tree demonstrates strong flexibility, and achieves higher IoU for events from 3 seconds to 630 seconds, which is the basis for subsequent understanding and localization of anomalies ($\text{mIoU}=0.52$ on UCF-Crime and $\text{mIoU}=0.64$ on XD-Violence).
  • Figure 2: The architecture of our proposed VADTree. The left side shows the construction of a hierarchical granularity-aware tree, which provides flexible multi-granularity characterization for the understanding and location of abnormal events. Then, as shown on the right, the description, reasoning, and refinement are implemented in a node-wise manner, and finally abnormal score integration is completed based on the topological relationship of the HGTree.
  • Figure 3: Qualitative results from VADTree on four test videos: two from UCF-Crime (top row) and two from XD-Violence (bottom row). The hierarchical video segment representations and corresponding anomaly scores are visualized alongside their key language explanations, with cyan and rose rectangles denoting coarse cluster and fine cluster nodes respectively. Each video's final anomaly scores (blue!80blue solid line) are computed by inter-cluster node correlation. Ground-truth anomalies are highlighted by red regions.
  • Figure 4: Performance comparison demonstrating the efficacy of inter-cluster correlation. The integration of hierarchical clusters in VADTree yields the highest AUC.
  • Figure 5: Influence of inter-cluster-correlation control coefficient $\beta$ on AUC.
  • ...and 2 more figures

Theorems & Definitions (2)

  • Theorem 1: Coverage Completeness
  • proof