Exploring the Magnitude-Shape Plot Framework for Anomaly Detection in Crowded Video Scenes

TL;DR

This work introduces a Magnitude-Shape (MS) Plot framework for video anomaly detection in crowded scenes by treating reconstruction residuals from autoencoders as multivariate functional data. By decomposing outlyingness into magnitude ($MO$) and shape ($VO$) components and combining them as $FO = \|MO\|^2 + VO$, the method provides a statistically principled, interpretable detector that leverages normal-only training data. The approach is integrated with both simple autoencoders and the advanced MAMA reconstruction model, yielding superior AUC and detection accuracy compared to univariate functional detectors and strong baselines on UCSD Ped2 and CUHK Avenue datasets. The results demonstrate the practical impact of MS-Plot analysis for robust anomaly detection in densely crowded videos, with clear visualization capabilities and potential for further unsupervised extensions.

Abstract

Detecting anomalies in crowded video scenes is critical for public safety, enabling timely identification of potential threats. This study explores video anomaly detection within a Functional Data Analysis framework, focusing on the application of the Magnitude-Shape (MS) Plot. Autoencoders are used to learn and reconstruct normal behavioral patterns from anomaly-free training data, resulting in low reconstruction errors for normal frames and higher errors for frames with potential anomalies. The reconstruction error matrix for each frame is treated as multivariate functional data, with the MS-Plot applied to analyze both magnitude and shape deviations, enhancing the accuracy of anomaly detection. Using its capacity to evaluate the magnitude and shape of deviations, the MS-Plot offers a statistically principled and interpretable framework for anomaly detection. The proposed methodology is evaluated on two widely used benchmark datasets, UCSD Ped2 and CUHK Avenue, demonstrating promising performance. It performs better than traditional univariate functional detectors (e.g., FBPlot, TVDMSS, Extremal Depth, and Outliergram) and several state-of-the-art methods. These results highlight the potential of the MS-Plot-based framework for effective anomaly detection in crowded video scenes.

Figures (4)

• Figure 1: Basic Autoencoder for Video Anomaly Detection.
• Figure 2: Sample frames from the UCSD Ped2 dataset showing anomalies: (a) Cyclist, (b) Van and cyclist, (c) Cyclist and skater, and (d) two cyclists. Red boxes highlight anomalous regions.
• Figure 3: Sample frames from the CUHK Avenue dataset showing anomalies: (a) running, (b) walking in unconventional directions, and (c) loitering.
• Figure 4: 3D MS-Plot representation for videos in the UCSD Ped2 testing set.