Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online Anomaly Detection over Live Social Video Streaming

Chengkun He, Xiangmin Zhou, Chen Wang, Iqbal Gondal, Jie Shao, Xun Yi

TL;DR

The paper tackles online anomaly detection in live social video streams by modeling the mutual influence between presenters and audiences with a Coupling LSTM (CLSTM) and a weighted reconstruction-score $RE_{IA}$. It integrates dual-stream prediction (influencer and audience) with decoders, and trains on normal data using a loss that combines JS divergence and MSE, enabling effective anomaly scoring through reconstruction errors. To make the system practical for streaming, it introduces dynamic updating, adaptive bound filtering (ADG and ADOS), and incremental computation for efficiency. Evaluations on four real-world datasets show superior AUROC and notable efficiency gains over state-of-the-art methods, with strong case-study support for live anomaly detection in online promotion and education contexts. The approach provides a scalable, real-time framework for monitoring anomalies in interactive online video content.

Abstract

Social video anomaly is an observation in video streams that does not conform to a common pattern of dataset's behaviour. Social video anomaly detection plays a critical role in applications from e-commerce to e-learning. Traditionally, anomaly detection techniques are applied to find anomalies in video broadcasting. However, they neglect the live social video streams which contain interactive talk, speech, or lecture with audience. In this paper, we propose a generic framework for effectively online detecting Anomalies Over social Video LIve Streaming (AOVLIS). Specifically, we propose a novel deep neural network model called Coupling Long Short-Term Memory (CLSTM) that adaptively captures the history behaviours of the presenters and audience, and their mutual interactions to predict their behaviour at next time point over streams. Then we well integrate the CLSTM with a decoder layer, and propose a new reconstruction error-based scoring function $RE_{IA}$ to calculate the anomaly score of each video segment for anomaly detection. After that, we propose a novel model update scheme that incrementally maintains CLSTM and decoder. Moreover, we design a novel upper bound and ADaptive Optimisation Strategy (ADOS) for improving the efficiency of our solution. Extensive experiments are conducted to prove the superiority of AOVLIS.

Online Anomaly Detection over Live Social Video Streaming

TL;DR

The paper tackles online anomaly detection in live social video streams by modeling the mutual influence between presenters and audiences with a Coupling LSTM (CLSTM) and a weighted reconstruction-score . It integrates dual-stream prediction (influencer and audience) with decoders, and trains on normal data using a loss that combines JS divergence and MSE, enabling effective anomaly scoring through reconstruction errors. To make the system practical for streaming, it introduces dynamic updating, adaptive bound filtering (ADG and ADOS), and incremental computation for efficiency. Evaluations on four real-world datasets show superior AUROC and notable efficiency gains over state-of-the-art methods, with strong case-study support for live anomaly detection in online promotion and education contexts. The approach provides a scalable, real-time framework for monitoring anomalies in interactive online video content.

Abstract

Social video anomaly is an observation in video streams that does not conform to a common pattern of dataset's behaviour. Social video anomaly detection plays a critical role in applications from e-commerce to e-learning. Traditionally, anomaly detection techniques are applied to find anomalies in video broadcasting. However, they neglect the live social video streams which contain interactive talk, speech, or lecture with audience. In this paper, we propose a generic framework for effectively online detecting Anomalies Over social Video LIve Streaming (AOVLIS). Specifically, we propose a novel deep neural network model called Coupling Long Short-Term Memory (CLSTM) that adaptively captures the history behaviours of the presenters and audience, and their mutual interactions to predict their behaviour at next time point over streams. Then we well integrate the CLSTM with a decoder layer, and propose a new reconstruction error-based scoring function to calculate the anomaly score of each video segment for anomaly detection. After that, we propose a novel model update scheme that incrementally maintains CLSTM and decoder. Moreover, we design a novel upper bound and ADaptive Optimisation Strategy (ADOS) for improving the efficiency of our solution. Extensive experiments are conducted to prove the superiority of AOVLIS.
Paper Structure (34 sections, 1 theorem, 23 equations, 13 figures, 4 tables)

This paper contains 34 sections, 1 theorem, 23 equations, 13 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation
  4. CLSTM-based Anomaly Detection
  5. Social Live Video Feature Extraction
  6. Action Recognition Feature
  7. Audience Interaction Feature
  8. The CLSTM Model
  9. The $LSTM_I$ for Modelling Influencer Behaviours
  10. The $LSTM_A$ for modelling audience interaction
  11. CLSTM Training Strategy
  12. Anomaly Detection
  13. Dynamic Updating
  14. Anomaly Detection Optimization
  15. ADG-based Dimensionality Reduction
  16. ...and 19 more sections

Key Result

Theorem 1

$RE_I^G$ is an upper bound of $\widetilde{RE_I}$ between the action recognition feature $f$ and its estimated feature $\hat{f}$.

Figures (13)

  • Figure 1: An example of real-life anomaly over live social video stream.
  • Figure 2: Overview of AOVLIS.
  • Figure 3: The application scenario.
  • Figure 4: CLSTM model.
  • Figure 5: Dynamic updating
  • ...and 8 more figures

Theorems & Definitions (4)

  • Definition 1
  • Definition 2
  • Theorem 1
  • proof