Mocap Anywhere: Towards Pairwise-Distance based Motion Capture in the Wild (for the Wild)

TL;DR

This work presents Mocap Anywhere, a camera-free motion capture system that reconstructs full-body 3D poses using only sparse pairwise distances from body-mounted UWB sensors. Central to the approach is WiP, a Transformer-based Refinement-Generative model that operates in the pairwise-distance space and learns to denoise noisy measurements, enabling real-time (≈50 FPS) high-fidelity motion capture in outdoor and wild environments. WiP supports both sparse-human and dense SMPL skeleton outputs, with shape-invariant variants and global-displacement anchors to recover global pose without prior morphology information. The method outperforms prior inertial-based baselines, demonstrates robust performance under noise and NLoS conditions, and extends to non-human subjects (e.g., animals) and wildlife scenarios through its generalization capabilities. Overall, WiP offers a portable, scalable, and general-purpose mocap solution with potential impact on animation, robotics, and biomechanics in unconstrained real-world settings.

Abstract

We introduce a novel motion capture system that reconstructs full-body 3D motion using only sparse pairwise distance (PWD) measurements from body-mounted(UWB) sensors. Using time-of-flight ranging between wireless nodes, our method eliminates the need for external cameras, enabling robust operation in uncontrolled and outdoor environments. Unlike traditional optical or inertial systems, our approach is shape-invariant and resilient to environmental constraints such as lighting and magnetic interference. At the core of our system is Wild-Poser (WiP for short), a compact, real-time Transformer-based architecture that directly predicts 3D joint positions from noisy or corrupted PWD measurements, which can later be used for joint rotation reconstruction via learned methods. WiP generalizes across subjects of varying morphologies, including non-human species, without requiring individual body measurements or shape fitting. Operating in real time, WiP achieves low joint position error and demonstrates accurate 3D motion reconstruction for both human and animal subjects in-the-wild. Our empirical analysis highlights its potential for scalable, low-cost, and general purpose motion capture in real-world settings.

Figures (14)

• Figure 1: Left: WiP is a Motion Capture Refinement-Generative Model (see \ref{['def:2']}). Given a clean sequence of pairwise distances $\hat{D}_{prev} \in \mathbb{R}^{w \times J \times J}$ up to timestamp $t-1$, and a sparse noisy measurement $\tilde{D}^{[t]} \in \mathbb{R}^{J_{\text{s}} \times J_{\text{s}}}$, the model predicts the next skeleton $\hat{P}^{[t]} \in \mathbb{R}^{J \times 3}$ in global coordinates. Simultaneously, it predicts the denoised pairwise distance matrix $\hat{D}^{[t]} \in \mathbb{R}^{J \times J}$ for a timestamp $t$, which is later used as input to predict pose $t+1$. To ensure consistency between the two output modalities, we apply a consistency loss $\mathcal{L}_{\text{cons}}$, in addition to separate reconstruction losses $\mathcal{L}_{\text{pd}}$ and $\mathcal{L}_{\text{dd}}$ for each modality. Right: The STJ-SA layers, inserted at the end of each Transformer block, enable the model to learn correspondences between joints across time, aligning per-joint trajectories and mitigating jitter caused by noisy measurements. We use $J_s = 6$ for the sparse configuration and $J = 24$ for the full SMPL skeleton representation.
• Figure 2: Reference Anchors Placement. To address global orientation of recorded subjects, we simulate a 2D global system by adding three sensors to the $\left|J_s\right|$-sized set. Mapping them to fixed spatial locations ($r_o \rightarrow \left(0,0,0\right), r_x \rightarrow \left(1,0,0\right)$, $r_y \rightarrow \left(0,1,0\right)$) and optimizing \ref{['eq:pd_loss', 'eq:dd_loss', 'loss:gravity']}, WiP predicts precise global positions.
• Figure 3: Full-Body Reconstruction (from DanceDB). Our method (blue), relying solely on PWD measurements, reconstructs full-body joint positions over time that are consistent with the ground truth (green).
• Figure 4: Qualitative Comparison on UIP-DB. By integrating SMPLify3D, we achieve strong performance, even compared to methods that predict joint rotations directly. Blue is our method, magenta is UIP, orange is UMotion, and ground truth is in green.
• Figure 5: SMPL Pose Estimation (from TotalCapture). When both PWD and inertial data are available, our method (blue) produces SMPL poses that closely match the ground truth (green).

Theorems & Definitions (2)

• definition 1: Refinement-Generative Model
• definition 2: Motion Capture Refinement-Generative Model