Person Parametric Physics-informed Representation for mmWave-based Human Pose Estimation

TL;DR

The paper tackles the persistent noise-versus-information problem in mmWave-based HPE by introducing PPPR, a parametric, Gaussian-primitives representation per joint that encodes both kinematic (position, velocity, orientation) and electromagnetic (scattering, Doppler) properties. PPPR is optimized via MmWave Human Parameterization (MHP) with dual physics-informed constraints: biomechanical kinematics and radar-physics electromagnetic consistency, augmented by a differentiable radar forward model that reconstructs a synthetic Heatmap for supervision. Across three public mmWave datasets and four HPE backbones, PPPR consistently improves accuracy and robustness, including cross-scene and cross-dataset generalization, while enabling radar-device agnostic deployment through calibration alone. The approach also extends to multi-person scenarios with explicit counting and inter-person separation constraints, achieving competitive performance with favorable efficiency (low parameter count and fast inference). These findings suggest PPPR as a practical, interpretable, and generalizable input representation that enhances deployability of mmWave-based HPE in diverse indoor environments.

Abstract

Millimeter-wave (mmWave) radar enables privacy-preserving, illumination-invariant Human Pose Estimation (HPE). However, current mmWave-based HPE systems face a signal-noise dilemma: Heatmaps retain human reflections but embed environmental clutter, while Point Clouds (PC) suppress noise through aggressive thresholding but discard informative human reflections, limiting robustness across environments and radar configurations. To address this intrinsic bottleneck, we introduce Person Parametric Physics-informed Representation (PPPR), a physics-informed parametric intermediate representation that replaces purely signal-level encodings with human-centric parameterization. PPPR models each human joint as a Gaussian primitive encoding both kinematic properties, which include position, velocity, orientation, and electromagnetic properties, which include scattering intensity and Doppler signature. These parameters enable optimization through a dual-constraint process: kinematic objectives enforce biomechanical consistency to suppress spatial artifacts, while electromagnetic objectives ensure adherence to mmWave propagation physics, decoupling input representations from non-human noise. Experiments across three mmWave-based HPE datasets with four HPE models demonstrate that replacing conventional inputs with PPPR consistently yields substantial accuracy gains. Furthermore, cross-scenes and cross-datasets experiments confirm PPPR's noise decoupling capability: models trained with PPPR maintain stable performance across diverse furniture arrangements and different radar chipsets, demonstrating its promising generalization capability in the challenging cross-dataset settings. Code will be released upon publication.

Figures (11)

• Figure 1: Person Parametric Physics-informed Representation (PPPR) processing flow. Input: Heatmap; Output: PPPR parameters; Additional Output: PPPR-enhanced Heatmap and PPPR-enhanced PC, for adaptability on existing Heatmap-based or PC-based HPE models.
• Figure 2: Millimeter wave radar signal processing pipeline: FMCW chirp transmission, three-stage FFT processing (Range, Doppler, Angle), and CFAR-based Point Cloud extraction.
• Figure 3: MmWave Human Parameterization (MHP), for a single person. Core components: Initialization (Sec. \ref{['sec:initialization']}), Radar Simulation (Sec. \ref{['sec:radar_sim']}), and Optimization (Sec. \ref{['sec:optimization']}). The multi-person extension is described in Sec. \ref{['sec:multiperson']}.
• Figure 4: MHP's adaptability to different radar FFT processing used on datasets.
• Figure 5: MmWave Human Parameterization (MHP) for multi-person scenarios. Two green-labeled Heatmaps are identical, both are the original input $H_{\mathrm{ori}}$. Key extensions include person counting to determine instance number and inter-person collision constraints to prevent skeletal overlap.