Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multimodal video analysis for crowd anomaly detection using open access tourism cameras

Alejandro Dionis-Ros, Joan Vila-Francés, Rafael Magdalena-Benedicto, Fernando Mateo, Antonio J. Serrano-López

TL;DR

The problem addressed is crowd anomaly detection in open webcam video data, with a privacy-preserving constraint. The authors convert video into two time-series, $D(t)$ for detections and $H(t)$ for heatmap saturation, and augment the data to enable robust temporal analysis. Their approach combines background subtraction, YOLOv8-based detections, STL decomposition with a daily period and weekly seasonality, and two detectors: a Trend Threshold for collective anomalies and SESD for point anomalies on the residuals, validated on the Morella webcam data with festivals in October 2023. The results demonstrate meaningful anomaly detection aligned across both time-series, highlighting festival-driven surges and identifying false positives, thereby offering a practical tool for tourism and security decision-making using open data while preserving individual privacy.

Abstract

In this article, we propose the detection of crowd anomalies through the extraction of information in the form of time series from video format using a multimodal approach. Through pattern recognition algorithms and segmentation, informative measures of the number of people and image occupancy are extracted at regular intervals, which are then analyzed to obtain trends and anomalous behaviors. Specifically, through temporal decomposition and residual analysis, intervals or specific situations of unusual behaviors are identified, which can be used in decision-making and improvement of actions in sectors related to human movement such as tourism or security. The application of this methodology on the webcam of Turisme Comunitat Valenciana in the town of Morella (Comunitat Valenciana, Spain) has provided excellent results. It is shown to correctly detect specific anomalous situations and unusual overall increases during the previous weekend and during the festivities in October 2023. These results have been obtained while preserving the confidentiality of individuals at all times by using measures that maximize anonymity, without trajectory recording or person recognition.

Multimodal video analysis for crowd anomaly detection using open access tourism cameras

TL;DR

The problem addressed is crowd anomaly detection in open webcam video data, with a privacy-preserving constraint. The authors convert video into two time-series, for detections and for heatmap saturation, and augment the data to enable robust temporal analysis. Their approach combines background subtraction, YOLOv8-based detections, STL decomposition with a daily period and weekly seasonality, and two detectors: a Trend Threshold for collective anomalies and SESD for point anomalies on the residuals, validated on the Morella webcam data with festivals in October 2023. The results demonstrate meaningful anomaly detection aligned across both time-series, highlighting festival-driven surges and identifying false positives, thereby offering a practical tool for tourism and security decision-making using open data while preserving individual privacy.

Abstract

In this article, we propose the detection of crowd anomalies through the extraction of information in the form of time series from video format using a multimodal approach. Through pattern recognition algorithms and segmentation, informative measures of the number of people and image occupancy are extracted at regular intervals, which are then analyzed to obtain trends and anomalous behaviors. Specifically, through temporal decomposition and residual analysis, intervals or specific situations of unusual behaviors are identified, which can be used in decision-making and improvement of actions in sectors related to human movement such as tourism or security. The application of this methodology on the webcam of Turisme Comunitat Valenciana in the town of Morella (Comunitat Valenciana, Spain) has provided excellent results. It is shown to correctly detect specific anomalous situations and unusual overall increases during the previous weekend and during the festivities in October 2023. These results have been obtained while preserving the confidentiality of individuals at all times by using measures that maximize anonymity, without trajectory recording or person recognition.
Paper Structure (24 sections, 2 equations, 15 figures)

This paper contains 24 sections, 2 equations, 15 figures.

Table of Contents

  1. Introduction
  2. Dataset
  3. Source of the data
  4. Data acquisition
  5. Data processing
  6. Generating the Time Series
  7. Detection Series
  8. Heatmap Saturation Percentage Series
  9. Methodology
  10. STL Decomposition
  11. Collective Anomalies - Trend Threshold
  12. Point Anomalies - SESD (Seasonal ESD)
  13. Analysis of results
  14. STL Decomposition
  15. Detection Series
  16. ...and 9 more sections

Figures (15)

  • Figure 1: Weekly distribution of median detection series
  • Figure 2: Weekly distribution of IQR detection series
  • Figure 3: Augmented detection series
  • Figure 4: Weekly distribution of the median of the heatmap series
  • Figure 5: Weekly distribution of the IQR of the heatmap series
  • ...and 10 more figures