Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Learning for Video Anomaly Detection: A Review

Peng Wu, Chengyu Pan, Yuting Yan, Guansong Pang, Peng Wang, Yanning Zhang

TL;DR

An extensive and comprehensive research review, covering the spectrum of five different categories of VAD, namely, semi-supervised, weakly supervised, fully supervised, unsupervised and open-set supervised VAD, and delve into the latest VAD works based on pre-trained large models, remedying the limitations of past reviews.

Abstract

Video anomaly detection (VAD) aims to discover behaviors or events deviating from the normality in videos. As a long-standing task in the field of computer vision, VAD has witnessed much good progress. In the era of deep learning, with the explosion of architectures of continuously growing capability and capacity, a great variety of deep learning based methods are constantly emerging for the VAD task, greatly improving the generalization ability of detection algorithms and broadening the application scenarios. Therefore, such a multitude of methods and a large body of literature make a comprehensive survey a pressing necessity. In this paper, we present an extensive and comprehensive research review, covering the spectrum of five different categories, namely, semi-supervised, weakly supervised, fully supervised, unsupervised and open-set supervised VAD, and we also delve into the latest VAD works based on pre-trained large models, remedying the limitations of past reviews in terms of only focusing on semi-supervised VAD and small model based methods. For the VAD task with different levels of supervision, we construct a well-organized taxonomy, profoundly discuss the characteristics of different types of methods, and show their performance comparisons. In addition, this review involves the public datasets, open-source codes, and evaluation metrics covering all the aforementioned VAD tasks. Finally, we provide several important research directions for the VAD community.

Deep Learning for Video Anomaly Detection: A Review

TL;DR

An extensive and comprehensive research review, covering the spectrum of five different categories of VAD, namely, semi-supervised, weakly supervised, fully supervised, unsupervised and open-set supervised VAD, and delve into the latest VAD works based on pre-trained large models, remedying the limitations of past reviews.

Abstract

Video anomaly detection (VAD) aims to discover behaviors or events deviating from the normality in videos. As a long-standing task in the field of computer vision, VAD has witnessed much good progress. In the era of deep learning, with the explosion of architectures of continuously growing capability and capacity, a great variety of deep learning based methods are constantly emerging for the VAD task, greatly improving the generalization ability of detection algorithms and broadening the application scenarios. Therefore, such a multitude of methods and a large body of literature make a comprehensive survey a pressing necessity. In this paper, we present an extensive and comprehensive research review, covering the spectrum of five different categories, namely, semi-supervised, weakly supervised, fully supervised, unsupervised and open-set supervised VAD, and we also delve into the latest VAD works based on pre-trained large models, remedying the limitations of past reviews in terms of only focusing on semi-supervised VAD and small model based methods. For the VAD task with different levels of supervision, we construct a well-organized taxonomy, profoundly discuss the characteristics of different types of methods, and show their performance comparisons. In addition, this review involves the public datasets, open-source codes, and evaluation metrics covering all the aforementioned VAD tasks. Finally, we provide several important research directions for the VAD community.
Paper Structure (64 sections, 16 equations, 10 figures, 5 tables)

This paper contains 64 sections, 16 equations, 10 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Notation and Taxonomy
  4. Datasets and Metrics
  5. Semi-supervised Video Anomaly Detection
  6. Model Input
  7. RGB
  8. Optical Flow
  9. Skeleton
  10. Hybrid
  11. Methodology
  12. Self-supervised Learning
  13. One-class Learning
  14. Interpretable Learning
  15. Network Architecture
  16. ...and 49 more sections

Figures (10)

  • Figure 1: Publications on VAD. Left: Google Scholar; Right: IEEE Xplore.
  • Figure 2: Performance development for semi/weakly supervised VAD tasks.
  • Figure 3: Comparisons of five supervised VAD tasks, i.e., fully supervised, semi-supervised, weakly supervised, unsupervised, and open-set supervised VAD.
  • Figure 4: The taxonomy of semi-supervised VAD. We provide a hierarchical taxonomy that organizes existing deep semi-supervised VAD models by model input, methodology, network architecture, refinement strategy, and model output into a systematic framework.
  • Figure 5: Flowchart of four classical explainable VAD methods.
  • ...and 5 more figures