Dual Conditioned Motion Diffusion for Pose-Based Video Anomaly Detection

TL;DR

The paper tackles pose-based video anomaly detection by unifying reconstruction- and prediction-based strategies within a diffusion framework. It introduces Dual Conditioned Motion Diffusion (DCMD), which jointly leverages conditioned motion and conditioned embedding, and uses a Motion Transformer denoiser to capture multi-layer correlations in motion spectra. A United Association Discrepancy (UAD) regularization, combining Gaussian kernel time association and self-attention global association, improves discriminability, while a mask completion strategy during inference enhances utilization of observed motions for predicting anomalies. Empirical results on four datasets show DCMD achieving state-of-the-art performance, with ablations confirming the necessity of each component and parameter analyses indicating robustness. The approach offers a principled way to fuse reconstruction and prediction in diffusion-based VAD, with clear implications for more reliable surveillance and safety applications.

Abstract

Video Anomaly Detection (VAD) is essential for computer vision research. Existing VAD methods utilize either reconstruction-based or prediction-based frameworks. The former excels at detecting irregular patterns or structures, whereas the latter is capable of spotting abnormal deviations or trends. We address pose-based video anomaly detection and introduce a novel framework called Dual Conditioned Motion Diffusion (DCMD), which enjoys the advantages of both approaches. The DCMD integrates conditioned motion and conditioned embedding to comprehensively utilize the pose characteristics and latent semantics of observed movements, respectively. In the reverse diffusion process, a motion transformer is proposed to capture potential correlations from multi-layered characteristics within the spectrum space of human motion. To enhance the discriminability between normal and abnormal instances, we design a novel United Association Discrepancy (UAD) regularization that primarily relies on a Gaussian kernel-based time association and a self-attention-based global association. Finally, a mask completion strategy is introduced during the inference stage of the reverse diffusion process to enhance the utilization of conditioned motion for the prediction branch of anomaly detection. Extensive experiments on four datasets demonstrate that our method dramatically outperforms state-of-the-art methods and exhibits superior generalization performance.

Figures (4)

• Figure 1: Illustration of video anomaly detection with normal instances (green) and abnormal instances (red).
• Figure 2: Overall architecture of the proposed method. The $H+F$ skeletal motion sequences are split into history motion sequences (red skeletal motion sequences) and future motion sequences (green skeletal motion sequences). The key point of our model is the dual conditioned motion diffusion, i.e., the hidden representation of the observed sequence obtained by the encoder and the complete sequence of observed sequences with added noise connected to the predicted future motion sequence.
• Figure 3: The architecture of the denoising network Motion Transformer. The green box on the right describes the details of the FFN module, and the blue box describes the details of the FiLM module.
• Figure 4: Graphical illustration of time association and global association. The blue curve is the normality score for a test segment of the CHUK dataset, and the red area indicates the time period in which the abnormal event occurred. Yellow curves indicate time association, and purple curves indicate global association.