LiveHPS++: Robust and Coherent Motion Capture in Dynamic Free Environment

TL;DR

LiveHPS++ advances LiDAR-based human motion capture in unconstrained, noisy environments by integrating a trajectory-aware body tracking module, a noise-resilient velocity predictor, and a kinematic-aware pose optimizer, all culminating in SMPL-based pose and shape estimation. The method enables robust, coherent global motion sequences from sequential noisy point clouds using a single LiDAR, significantly outperforming prior state-of-the-art methods on multiple datasets, including the newly proposed NoiseMotion synthetic dataset. Key contributions include the three-module architecture, an effective distillation strategy to preserve trajectory information, and a comprehensive NoiseMotion dataset that simulates realistic human–object interactions with dynamic and static noise. These advances address practical needs for outdoor, real-time motion capture and hold promise for applications in robotics, AR/VR, and dynamic scene understanding.

Abstract

LiDAR-based human motion capture has garnered significant interest in recent years for its practicability in large-scale and unconstrained environments. However, most methods rely on cleanly segmented human point clouds as input, the accuracy and smoothness of their motion results are compromised when faced with noisy data, rendering them unsuitable for practical applications. To address these limitations and enhance the robustness and precision of motion capture with noise interference, we introduce LiveHPS++, an innovative and effective solution based on a single LiDAR system. Benefiting from three meticulously designed modules, our method can learn dynamic and kinematic features from human movements, and further enable the precise capture of coherent human motions in open settings, making it highly applicable to real-world scenarios. Through extensive experiments, LiveHPS++ has proven to significantly surpass existing state-of-the-art methods across various datasets, establishing a new benchmark in the field.

Figures (6)

• Figure 1: Visualization of the motion capture performance of LiveHPS++ in a real-time captured scenario with complex human-object interaction. The left exhibits images for reference, the middle shows the noised point clouds (top) and our corresponding mesh model results (bottom). We zoom in some cases on the right for clearer demonstration, where point clouds are drawn in white.
• Figure 2: The pipeline of LiveHPS++. It consists of three primary modules, including a trajectory-guided body tracker to predict the human joint and translation, a noise-insensitive velocity predictor to regress the velocity, and the kinematic-aware pose optimizer to enhance the accuracy and coherence of results. Finally, we use SMPL solver to regress the parameters of human poses and shape. Detailed network structure of three modules is also shown under the upper pipeline.
• Figure 3: Normalized point cloud. The light green point represents the human root positions, while the dark green point represents the origin of coordinate axis after normalization. The sequential point cloud on the left without noise can be normalized to obtain a relatively stable data distribution, while the data on the right exhibits a more jittery data distribution after normalization due to noise interference.
• Figure 4: The NoiseMotion dataset simulation pipeline, integrating dynamic human motion and static object noise to simulate real-world human-object interactions.
• Figure 5: Qualitative comparisons. The point cloud matches the result better, representing more accurate estimation for pose, shape, and translation. Each point in the visualization of coherence evaluation represents the frame-wise global human translations in the bird's-eye view.