Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Real-time 3D human action recognition based on Hyperpoint sequence

Xing Li, Qian Huang, Zhijian Wang, Zhenjie Hou, Tianjin Yang, Zhuang Miao

TL;DR

A novel type of point data, hyperpoint, is defined to better describe the temporally changing human appearances and a theoretical foundation is provided to clarify the information equivalence property for converting point cloud sequences into hyperpoint sequences.

Abstract

Real-time 3D human action recognition has broad industrial applications, such as surveillance, human-computer interaction, and healthcare monitoring. By relying on complex spatio-temporal local encoding, most existing point cloud sequence networks capture spatio-temporal local structures to recognize 3D human actions. To simplify the point cloud sequence modeling task, we propose a lightweight and effective point cloud sequence network referred to as SequentialPointNet for real-time 3D action recognition. Instead of capturing spatio-temporal local structures, SequentialPointNet encodes the temporal evolution of static appearances to recognize human actions. Firstly, we define a novel type of point data, Hyperpoint, to better describe the temporally changing human appearances. A theoretical foundation is provided to clarify the information equivalence property for converting point cloud sequences into Hyperpoint sequences. Secondly, the point cloud sequence modeling task is decomposed into a Hyperpoint embedding task and a Hyperpoint sequence modeling task. Specifically, for Hyperpoint embedding, the static point cloud technology is employed to convert point cloud sequences into Hyperpoint sequences, which introduces inherent frame-level parallelism; for Hyperpoint sequence modeling, a Hyperpoint-Mixer module is designed as the basic building block to learning the spatio-temporal features of human actions. Extensive experiments on three widely-used 3D action recognition datasets demonstrate that the proposed SequentialPointNet achieves competitive classification performance with up to 10X faster than existing approaches.

Real-time 3D human action recognition based on Hyperpoint sequence

TL;DR

A novel type of point data, hyperpoint, is defined to better describe the temporally changing human appearances and a theoretical foundation is provided to clarify the information equivalence property for converting point cloud sequences into hyperpoint sequences.

Abstract

Real-time 3D human action recognition has broad industrial applications, such as surveillance, human-computer interaction, and healthcare monitoring. By relying on complex spatio-temporal local encoding, most existing point cloud sequence networks capture spatio-temporal local structures to recognize 3D human actions. To simplify the point cloud sequence modeling task, we propose a lightweight and effective point cloud sequence network referred to as SequentialPointNet for real-time 3D action recognition. Instead of capturing spatio-temporal local structures, SequentialPointNet encodes the temporal evolution of static appearances to recognize human actions. Firstly, we define a novel type of point data, Hyperpoint, to better describe the temporally changing human appearances. A theoretical foundation is provided to clarify the information equivalence property for converting point cloud sequences into Hyperpoint sequences. Secondly, the point cloud sequence modeling task is decomposed into a Hyperpoint embedding task and a Hyperpoint sequence modeling task. Specifically, for Hyperpoint embedding, the static point cloud technology is employed to convert point cloud sequences into Hyperpoint sequences, which introduces inherent frame-level parallelism; for Hyperpoint sequence modeling, a Hyperpoint-Mixer module is designed as the basic building block to learning the spatio-temporal features of human actions. Extensive experiments on three widely-used 3D action recognition datasets demonstrate that the proposed SequentialPointNet achieves competitive classification performance with up to 10X faster than existing approaches.
Paper Structure (25 sections, 7 equations, 5 figures, 7 tables)

This paper contains 25 sections, 7 equations, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Static Point Cloud Modeling
  4. Point Cloud Sequence-based 3D Human Action Recognition
  5. METHODOLOGY
  6. Hyperpoint Sequence
  7. Hyperpoint Embedding Module
  8. Hyperpoint-Mixer Module
  9. Space dislocation layer
  10. Hyperpoint-mixing layer
  11. Experiments
  12. Datasets
  13. Comparison with the State-of-the-art Methods
  14. NTU RGB+D 60 dataset
  15. NTU RGB+D 120 dataset
  16. ...and 10 more sections

Figures (5)

  • Figure 1: SequentialPointNet decomposes the complex point cloud sequence modeling task into a static point cloud technology-based Hyperpoint embedding task and a simple Hyperpoint sequence modeling task.
  • Figure 2: SequentialPointNet contains a Hyperpoint embedding module, a Hyperpoint-Mixer module, and a classifier head.
  • Figure 3: The comparison of the three types of point data.
  • Figure 4: Left: Hyperpoint-Mixer module. Right: Space dislocation layer.
  • Figure 5: The computation graph of SequentialPointNet.