Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Euclidean: Dual-Space Representation Learning for Weakly Supervised Video Violence Detection

Jiaxu Leng, Zhanjie Wu, Mingpi Tan, Yiran Liu, Ji Gan, Haosheng Chen, Xinbo Gao

TL;DR

A novel Dual-Space Representation Learning (DSRL) method for weakly supervised VVD to utilize the strength of both Euclidean and hyperbolic geometries, capturing the visual features of events while also exploring the intrinsic relations between events, thereby enhancing the discriminative capacity of the features.

Abstract

While numerous Video Violence Detection (VVD) methods have focused on representation learning in Euclidean space, they struggle to learn sufficiently discriminative features, leading to weaknesses in recognizing normal events that are visually similar to violent events (\emph{i.e.}, ambiguous violence). In contrast, hyperbolic representation learning, renowned for its ability to model hierarchical and complex relationships between events, has the potential to amplify the discrimination between visually similar events. Inspired by these, we develop a novel Dual-Space Representation Learning (DSRL) method for weakly supervised VVD to utilize the strength of both Euclidean and hyperbolic geometries, capturing the visual features of events while also exploring the intrinsic relations between events, thereby enhancing the discriminative capacity of the features. DSRL employs a novel information aggregation strategy to progressively learn event context in hyperbolic spaces, which selects aggregation nodes through layer-sensitive hyperbolic association degrees constrained by hyperbolic Dirichlet energy. Furthermore, DSRL attempts to break the cyber-balkanization of different spaces, utilizing cross-space attention to facilitate information interactions between Euclidean and hyperbolic space to capture better discriminative features for final violence detection. Comprehensive experiments demonstrate the effectiveness of our proposed DSRL.

Beyond Euclidean: Dual-Space Representation Learning for Weakly Supervised Video Violence Detection

TL;DR

A novel Dual-Space Representation Learning (DSRL) method for weakly supervised VVD to utilize the strength of both Euclidean and hyperbolic geometries, capturing the visual features of events while also exploring the intrinsic relations between events, thereby enhancing the discriminative capacity of the features.

Abstract

While numerous Video Violence Detection (VVD) methods have focused on representation learning in Euclidean space, they struggle to learn sufficiently discriminative features, leading to weaknesses in recognizing normal events that are visually similar to violent events (\emph{i.e.}, ambiguous violence). In contrast, hyperbolic representation learning, renowned for its ability to model hierarchical and complex relationships between events, has the potential to amplify the discrimination between visually similar events. Inspired by these, we develop a novel Dual-Space Representation Learning (DSRL) method for weakly supervised VVD to utilize the strength of both Euclidean and hyperbolic geometries, capturing the visual features of events while also exploring the intrinsic relations between events, thereby enhancing the discriminative capacity of the features. DSRL employs a novel information aggregation strategy to progressively learn event context in hyperbolic spaces, which selects aggregation nodes through layer-sensitive hyperbolic association degrees constrained by hyperbolic Dirichlet energy. Furthermore, DSRL attempts to break the cyber-balkanization of different spaces, utilizing cross-space attention to facilitate information interactions between Euclidean and hyperbolic space to capture better discriminative features for final violence detection. Comprehensive experiments demonstrate the effectiveness of our proposed DSRL.
Paper Structure (18 sections, 1 theorem, 24 equations, 7 figures, 4 tables)

This paper contains 18 sections, 1 theorem, 24 equations, 7 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Methodology
  5. Hyperbolic Energy-constrained Graph Convolutional Network Module (HE-GCN)
  6. Dual-Space Interaction Module(DSI)
  7. Experiments
  8. Experiments Setup
  9. Comparisons with State-of-the-art Methods
  10. Ablation Studies
  11. Qualitative Results
  12. Conclusions
  13. Proof of Theorem. \ref{['thm:1']}
  14. Technical details of the DSRL
  15. Experimental Details
  16. ...and 3 more sections

Key Result

Theorem 1

$\forall \textbf{x} \in \mathbb{L}^{n}$, $\textbf{M}\in \mathbb{R}^{(m+1)\times(n+1)}$, we have $f_{x}(\textbf{M})\textbf{x} \in \mathbb{L}_{K}^{m}$.

Figures (7)

  • Figure 1: (a) Hierarchical diagram in Video Violence Detection (VVD). (b) Our DSRL enhances the detection of ambiguous violence by combining Euclidean and Hyperbolic spaces to balance visual feature expression and hierarchical event relations.
  • Figure 2: A conceptual diagram of our DSRL.
  • Figure 3: t-SNE visualization of vanilla and DSRL features for the test video on XD-Violence.
  • Figure 4: Frame-level scores and violence localization examples for the test video from XD-Violence dataset.
  • Figure 5: Some visual results of DSRL in the context of ambiguous violence.
  • ...and 2 more figures

Theorems & Definitions (1)

  • Theorem 1