DA-Flow: Dual Attention Normalizing Flow for Skeleton-based Video Anomaly Detection

TL;DR

DA-Flow tackles skeleton-based video anomaly detection by coupling a lightweight Dual Attention Module (DAM) with a Glow-like normalizing flow. GCN handles local skeletal relations while DAM captures global cross-dimension semantics through Skeleton Attention and Frame Attention, informing a multiscale invertible transform that optimizes $ ext{log } p_X(x)$ via $ ext{log } p_X(x) = ext{log } p_Z(z) - ext{log } | ext{det } J_T(z)|$. The approach achieves state-of-the-art or competitive micro AUC on five benchmarks with an extremely small parameter footprint (about $0.488$K) and demonstrates robustness to noise and data contamination, including surprising zero-training effectiveness with random projections. These results argue for focusing on the statistical characteristics of normal skeleton data for SVAD, offering a fast, privacy-preserving solution suitable for real-time applications.

Abstract

Cooperation between temporal convolutional networks (TCN) and graph convolutional networks (GCN) as a processing module has shown promising results in skeleton-based video anomaly detection (SVAD). However, to maintain a lightweight model with low computational and storage complexity, shallow GCN and TCN blocks are constrained by small receptive fields and a lack of cross-dimension interaction capture. To tackle this limitation, we propose a lightweight module called the Dual Attention Module (DAM) for capturing cross-dimension interaction relationships in spatio-temporal skeletal data. It employs the frame attention mechanism to identify the most significant frames and the skeleton attention mechanism to capture broader relationships across fixed partitions with minimal parameters and flops. Furthermore, the proposed Dual Attention Normalizing Flow (DA-Flow) integrates the DAM as a post-processing unit after GCN within the normalizing flow framework. Simulations show that the proposed model is robust against noise and negative samples. Experimental results show that DA-Flow reaches competitive or better performance than the existing state-of-the-art (SOTA) methods in terms of the micro AUC metric with the fewest number of parameters. Moreover, we found that even without training, simply using random projection without dimensionality reduction on skeleton data enables substantial anomaly detection capabilities.

Figures (12)

• Figure 1: The comparison between DA-Flow with State-of-the-Art Skeleton-based Video Anomaly Detection methods (STG-NF stgnf, MOCODAD mocodad, COSKAD COSKAD,TrajREC stergiou2024traj) and GiCiSAD karami2024gici on UBnormal dataset. Our method surpasses these methods on the AUC metric with the fewest number of parameters.
• Figure 2: Overview of the DA-flow Methodology. The process initiates with pose estimation and tracking of input video data. Each pose sequence is then individually processed by the DA-flow model. Training involves learning a bijective mapping from the data distribution $p_X(x)$ (pose sequences) to a latent Gaussian prior $p_Z(z)$, achieved by minimizing the negative log-likelihood of the data. This utilizes the invertibility of the architecture along with the change of variables formula. During inference, the probability of each pose sequence is assessed, and the frame score is determined by the sequence with the lowest log-likelihood score.
• Figure 3: Multiscale architecture with four levels as introduced in 29NVP. First, the entire input $x$ is transformed by $T_1$. The result is then split up into two parts of which one of them is factored out immediately and the other one is further processed by $T_2$. This process is repeated a few times until the desired depth is reached. The input is drawn in green, intermediate results are in red, and the components of the final variable $z$ are yellow.
• Figure 4: Schematic diagram of the DA Flow includes Actnorm, permutation, and affine coupling layers in each transformation unit. A transformation unit begins with a GCN that extracts local information from a heuristic partition of the human skeleton. The subsequent DAM consists of two main branches: the upper branch dedicated to extracting Skeleton Attention, and the lower branch, which splits further to concurrently extract Frame Attention.
• Figure 5: The figure for DAM: DAM captures the global information across all input frames and cross-GCN-partition: The red and green boxes represent the most noteworthy skeleton nodes called Skeleton Attention and the orange box denotes the frame that is most noteworthy for determining the anomaly called Frame Attention.