Deformation-Aware Observation Modeling for Radar-Based Human Sensing via 3D Scan-Depth Sequence Fusion
Guangqi Shi, Kimitaka Sumi, Takuya Sakamoto
TL;DR
This work tackles the challenge of respiration-induced non-rigid torso deformation in radar-based sensing by introducing a deformation-aware observation framework that fuses static high-resolution 3D scanner data with time-series depth-camera frames via non-rigid CPD registration. It models frame-wise electromagnetic scattering with the physical optics approximation and reconstructs intermediate-frequency radar signals to generate physics-grounded radar observations. Experimental validation across three participants demonstrates improved alignment with measured displacements and greater robustness under low-signal conditions compared with depth-sequence-only methods, including a noteworthy I–Q magnitude correlation of 0.789 in a favorable single-site scenario. While offering a more realistic interpretation of radar measurements, the approach faces challenges in computational cost and temporal smoothing that warrant future refinement for practical deployment.
Abstract
Non-contact radar-based human sensing is often interpreted using simplified motion assumptions. However, respiration induces non-rigid surface deformation of the human body that impacts electromagnetic wave scattering and can degrade the robustness of measurements. To address this, we propose a surface-deformation-aware observation model for radar-based human sensing that fuses static high-resolution three-dimensional scanner measurements with temporal depth camera data to represent time-varying human surface geometry. Non-rigid registration using the coherent point drift algorithm is employed to align a static template with dynamic depth frames. Frame-wise electromagnetic scattering is subsequently computed using the physical optics approximation, allowing the reconstruction of intermediate-frequency radar signals that emulate radar observations. Validation against experimental radar data demonstrated that the proposed model exhibited greater robustness than a depth-sequence-only model under low-signal-quality conditions involving complex surface dynamics and multiple reflective sites. For two participants, the proposed model achieved higher Pearson correlation coefficients of 0.943 and 0.887 between model-derived and experimentally measured displacement waveforms, compared with 0.868 and 0.796 for the depth-sequence-only model. Furthermore, in a favorable case characterized by a single relatively-stationary reflective site, the proposed method achieved a correlation coefficient of 0.789 between model-derived and experimentally measured in-phase-quadrature magnitude variations. These results suggest that our sensor-fusion-based deformation-aware observation modeling can realistically reproduce radar observations and provide physically grounded insights into the interpretation of radar measurement variations.