End-to-End Motion Capture from Rigid Body Markers with Geodesic Loss

TL;DR

The paper introduces Rigid Body Markers (RBMs) as a sparse, unambiguous 6-DoF input unit for marker-based MoCap and demonstrates end-to-end SMPL parameter regression using a geodesic loss on SO(3). By synthesizing RBM data from AMASS and aligning real RBMs via a T-pose calibration, the approach achieves state-of-the-art accuracy while significantly reducing computation compared to optimization-based methods. The key contributions are the RBM hardware concept and the geodesic loss as a rotation-aware training objective, which together enable real-time, high-fidelity motion capture suitable for graphics, VR, and biomechanics. The method is validated on synthetic AMASS data and real-world Vicon data, and ablation studies confirm the benefits of pose normalization and the geodesic loss. Overall, the work offers a practical framework that matches or exceeds dense-marker performance with far lower setup complexity and computational cost.

Abstract

Marker-based optical motion capture (MoCap), while long regarded as the gold standard for accuracy, faces practical challenges, such as time-consuming preparation and marker identification ambiguity, due to its reliance on dense marker configurations, which fundamentally limit its scalability. To address this, we introduce a novel fundamental unit for MoCap, the Rigid Body Marker (RBM), which provides unambiguous 6-DoF data and drastically simplifies setup. Leveraging this new data modality, we develop a deep-learning-based regression model that directly estimates SMPL parameters under a geodesic loss. This end-to-end approach matches the performance of optimization-based methods while requiring over an order of magnitude less computation. Trained on synthesized data from the AMASS dataset, our end-to-end model achieves state-of-the-art accuracy in body pose estimation. Real-world data captured using a Vicon optical tracking system further demonstrates the practical viability of our approach. Overall, the results show that combining sparse 6-DoF RBM with a manifold-aware geodesic loss yields a practical and high-fidelity solution for real-time MoCap in graphics, virtual reality, and biomechanics.

Figures (6)

• Figure 1: Schematic of the Rigid Body Marker (RBM) and its sparse placement on the human body for capturing full-body motion.
• Figure 2: Pipeline of the proposed method. The upper part shows synthetic-data generation and network training; the lower part shows inference, where real inputs are calibrated and fed into the trained network to produce the final output.
• Figure 3: Illustration of the local coordinate frame construction.
• Figure 4: Overview of the network architecture and loss design. The network predicts SMPL parameters $(\hat{\beta}, \hat{\theta}, \hat{\gamma})$ from 6-DoF RBM input. The pose loss $\mathcal{L}_{\theta}$ employs the geodesic loss, while the shape loss $\mathcal{L}_{\beta}$ and translation loss $\mathcal{L}_{\gamma}$ use MSE loss. The total loss is computed as a weighted sum of these components.
• Figure 5: Performance comparison of RBM configurations against the dense-marker baseline. The performance of seven RBM setups (detailed in Table \ref{['tab:rbm_config']}) is plotted, with the dense-marker system's result shown as a horizontal dashed line. Performance is measured by MPJPE (mm), PA-MPJPE (mm), and MPJAE (deg).