WaveMan: mmWave-Based Room-Scale Human Interaction Perception for Humanoid Robots

TL;DR

WaveMan tackles privacy-preserving, room-scale humanoid-robot interaction by introducing a spatially adaptive mmWave perception pipeline that aligns radar observations to a canonical space, enhances sparse spectrograms, and employs a dual-branch attention-based recognizer. The system combines geometric alignment, unpaired spectrogram translation, and DBCA-based recognition to achieve strong cross-position generalization, enabling reliable gesture understanding across unconstrained user locations. Experimental results demonstrate substantial gains in unseen-position and random-position accuracy, with rapid inference suitable for real-time robot control. This work advances practical, privacy-aware HRI by delivering robust room-scale sensing that generalizes beyond fixed viewpoints and distances, paving the way for multimodal integration in household robotics.

Abstract

Reliable humanoid-robot interaction (HRI) in household environments is constrained by two fundamental requirements, namely robustness to unconstrained user positions and preservation of user privacy. Millimeter-wave (mmWave) sensing inherently supports privacy-preserving interaction, making it a promising modality for room-scale HRI. However, existing mmWave-based interaction-sensing systems exhibit poor spatial generalization at unseen distances or viewpoints. To address this challenge, we introduce WaveMan, a spatially adaptive room-scale perception system that restores reliable human interaction sensing across arbitrary user positions. WaveMan integrates viewpoint alignment and spectrogram enhancement for spatial consistency, with dual-channel attention for robust feature extraction. Experiments across five participants show that, under fixed-position evaluation, WaveMan achieves the same cross-position accuracy as the baseline with five times fewer training positions. In random free-position testing, accuracy increases from 33.00% to 94.33%, enabled by the proposed method. These results demonstrate the feasibility of reliable, privacy-preserving interaction for household humanoid robots across unconstrained user positions.

Figures (9)

• Figure 1: Spatially adaptive room-scale interaction scenario. WaveMan aligns observations from different user spatial positions into a unified perception space to mitigate spatial inconsistencies.
• Figure 2: Overview of the proposed spatially adaptive interaction framework. (a) Radar point-cloud data captured under diverse positional configurations are spatially aligned and transformed into spectrogram representations. (b) Sparse spectrograms are enhanced and fused with dense spectra to obtain robust gesture representations. (c) The recognized gestures are mapped to corresponding humanoid robot behaviors, enabling reliable human--robot interaction across unconstrained user positions.
• Figure 3: Spatially adaptive point cloud alignment. (a) shows the geometric relationship between the real radar position $\mathrm{O_a}$, the virtual canonical radar position $\mathrm{O_b}$, and the observed point $\mathrm{P}$, where azimuth and elevation offsets are compensated through sequential rotations and translation. (b) presents an example point cloud before and after alignment, illustrating reduced angular distortion and a more compact spatial distribution for subsequent spectrogram generation.
• Figure 4: Architecture of the proposed spectrogram enhancement and recognition network. (a) Unpaired spectrogram enhancement pipeline, where sparse spectral inputs are translated into dense representations using an Enhancer–Reducer pair supervised by two discriminators. (b) Internal network components, including the Enhancer/Reducer architectures, PatchGAN-based discriminators, and a compact CNN equipped with the proposed Dual-Branch Channel Attention (DBCA) module for generalization-oriented gesture recognition.
• Figure 5: Sampled examples of spectrograms and quantitative metrics before/after alignment (top) and enhancement (bottom).