Body and Head Orientation Estimation from Low-Resolution Point Clouds in Surveillance Settings

TL;DR

This work demonstrates that body and head orientations can be accurately estimated from low-resolution, privacy-preserving LiDAR point clouds collected from a surveillance viewpoint. It introduces a two-part system: ellipse-based body orientation from a birds-eye projection and a geometry-guided feature set regressed by an ensemble of neural networks for head yaw, enhanced by head-position corrections and RF-based feature selection. On static data, body orientation achieves about 5.2° MAE, while on conversation data involving ASD and NT participants, head orientation attains about 13.7° MAE, with clear advantages over high-resolution methods when applied to low-resolution LiDAR. Beyond pose estimation, the study quantifies triadic interaction differences between autistic and neurotypical adults, suggesting potential coaching and behavioral analysis applications, while acknowledging limitations like small sample size and controlled settings.

Abstract

We propose a system that estimates people's body and head orientations using low-resolution point cloud data from two LiDAR sensors. Our models make accurate estimations in real-world conversation settings where subjects move naturally with varying head and body poses, while seated around a table. The body orientation estimation model uses ellipse fitting while the head orientation estimation model combines geometric feature extraction with an ensemble of neural network regressors. Our models achieve a mean absolute estimation error of 5.2 degrees for body orientation and 13.7 degrees for head orientation. Compared to other body/head orientation estimation systems that use RGB cameras, our proposed system uses LiDAR sensors to preserve user privacy, while achieving comparable accuracy. Unlike other body/head orientation estimation systems, our sensors do not require a specified close-range placement in front of the subject, enabling estimation from a surveillance viewpoint which produces low-resolution data. This work is the first to attempt head orientation estimation using point clouds in a low-resolution surveillance setting. We compare our model to two state-of-the-art head orientation estimation models that are designed for high-resolution point clouds, which yield higher estimation errors on our low-resolution dataset. We also present an application of head orientation estimation by quantifying behavioral differences between neurotypical and autistic individuals in triadic (three-way) conversations. Significance tests show that autistic individuals display significantly different behavior compared to neurotypical individuals in distributing attention between conversational parties, suggesting that the approach could be a component of a behavioral analysis or coaching system.

Figures (5)

• Figure 1: System overview.
• Figure 2: Conversation Setups
• Figure 3: Least squares ellipse fitting for body orientation estimation via the long axis of the fitted ellipse, which also determines the four quadrants of the head relative to the body. Light blue points are projected upper body points (shoulders and chest); dark blue, red, purple and green points are projected head points, representing the four quadrants, in order. The first and second quadrants represent the front-left and front-right sides of the head, respectively; while the third and fourth quadrants represent the back-left and back-right sides of the head. (a) Head orientation label unknown; point cloud belongs to a head movement that starts at 0 degrees and moves to the left (b) Head orientation labeled as 35 degrees to the left.
• Figure 4: Evolution of mean absolute estimation error with the Random Forest Recursive Feature Elimination procedure. The initial MAE with 103 features is 15.72 degrees, whereas the MAE with only 1 feature left in the feature space is 25.61 degrees. The optimal feature set contains 42 features and leads to an MAE of 13.73 degrees.
• Figure 5: Performance of optimal feature set and its subsets (PC = Principal Component).