Continuous Human Action Recognition for Human-Machine Interaction: A Review

TL;DR

A large body of recent related work in the literature is thoroughly analysed, explain, and compare action segmentation methods and provides details on the feature extraction and learning strategies that are used on most state-of-the-art methods.

Abstract

With advances in data-driven machine learning research, a wide variety of prediction models have been proposed to capture spatio-temporal features for the analysis of video streams. Recognising actions and detecting action transitions within an input video are challenging but necessary tasks for applications that require real-time human-machine interaction. By reviewing a large body of recent related work in the literature, we thoroughly analyse, explain and compare action segmentation methods and provide details on the feature extraction and learning strategies that are used on most state-of-the-art methods. We cover the impact of the performance of object detection and tracking techniques on human action segmentation methodologies. We investigate the application of such models to real-world scenarios and discuss several limitations and key research directions towards improving interpretability, generalisation, optimisation and deployment.

Figures (23)

• Figure 1: Discrete and Continuous Action Recognition Comparison.
• Figure 2: An Example of a Continuous Action Video Sequence: The frames that include action transitions are often labelled as "background"(BCKG) or "Null".
• Figure 3: Action Segmentation Pipeline.
• Figure 4: Temporal Convolutional Networks (TCN).
• Figure 5: Dilation operations with different dilation factors ($d$) with $kernel_size = 3$. In the case where $d=1$, 3 adjacent input elements are chosen to compute a particular output element by setting the receptive field size to 3. When $d=2$ the receptive field is increased to length 5. When $d=4$, the receptive field expands to a length of 9.