Wave-Former: Through-Occlusion 3D Reconstruction via Wireless Shape Completion

TL;DR

Wave-Former tackles the challenge of reconstructing complete 3D geometry for fully occluded objects using millimeter-wave signals. By embedding mmWave physics into a transformer-based shape completion framework and implementing a three-stage real-world inference pipeline, it trains entirely on synthetic data yet generalizes to real measurements. Key innovations include a specularity-aware inductive bias, reflection-dependent visibility, and a joint denoising-completion objective, augmented by entropy-guided surface selection to pick the best reconstruction. The method achieves state-of-the-art recall (72%) with high precision (85%) on diverse objects, demonstrating strong potential for robotics, AR, and logistics applications that require through-occlusion perception.

Abstract

We present Wave-Former, a novel method capable of high-accuracy 3D shape reconstruction for completely occluded, diverse, everyday objects. This capability can open new applications spanning robotics, augmented reality, and logistics. Our approach leverages millimeter-wave (mmWave) wireless signals, which can penetrate common occlusions and reflect off hidden objects. In contrast to past mmWave reconstruction methods, which suffer from limited coverage and high noise, Wave-Former introduces a physics-aware shape completion model capable of inferring full 3D geometry. At the heart of Wave-Former's design is a novel three-stage pipeline which bridges raw wireless signals with recent advancements in vision-based shape completion by incorporating physical properties of mmWave signals. The pipeline proposes candidate geometric surfaces, employs a transformer-based shape completion model designed specifically for mmWave signals, and finally performs entropy-guided surface selection. This enables Wave-Former to be trained using entirely synthetic point-clouds, while demonstrating impressive generalization to real-world data. In head-to-head comparisons with state-of-the-art baselines, Wave-Former raises recall from 54% to 72% while maintaining a high precision of 85%.

Figures (5)

• Figure 1: Comparison between Wave-Former and state-of-the-art mmWave reconstruction baselines.
• Figure 2: Specular reflection.Unlike visible light, which primarily scatters, mmWave signals undergo primarily specular (mirror-like) reflections.
• Figure 3: mmWave Reconstruction Pipeline.a) Wave-Former's physics-aware training pipeline incorporates physical properties through a specularity-aware inductive bias, reflection-dependent visibility, and joint refinement and completion framework to enable training on entirely synthetic data. b) Wave-Former's real-world inference process leverages a three-stage pipeline to reconstruct a complete 3D object from real mmWave signals.
• Figure 4: Specularity-Aware Inductive Bias.Specularity-aware inductive bias generates training partials (e) that resemble real mmWave visibility (d), unlike standard masking used in vision-based models (c).
• Figure 5: Qualitative Results.Visual comparison of mmWave 3D reconstruction on real-world, fully occluded objects. State-of-the-art baselines suffer from artifacts such as high noise and limited coverage, while Wave-Former consistently reconstructs shapes with high fidelity.