Rascene: High-Fidelity 3D Scene Imaging with mmWave Communication Signals

Abstract

Robust 3D environmental perception is critical for applications such as autonomous driving and robot navigation. However, optical sensors such as cameras and LiDAR often fail under adverse conditions, including smoke, fog, and non-ideal lighting. Although specialized radar systems can operate in these environments, their reliance on bespoke hardware and licensed spectrum limits scalability and cost-effectiveness. This paper introduces Rascene, an integrated sensing and communication (ISAC) framework that leverages ubiquitous mmWave OFDM communication signals for 3D scene imaging. To overcome the sparse and multipath-ambiguous nature of individual radio frames, Rascene performs multi-frame, spatially adaptive fusion with confidence-weighted forward projection, enabling the recovery of geometric consensus across arbitrary poses. Experimental results demonstrate that our method reconstructs 3D scenes with high precision, offering a new pathway toward low-cost, scalable, and robust 3D perception.

Figures (16)

• Figure 1: High-fidelity 3D imaging generated by Rascene from mmWave communication signals. We show that OFDM communication signals can support high-fidelity 3D imaging on a single device. Our multi-frame RF fusion suppresses multipath artifacts and integrates sparse observations into a complete 3D scene estimate (blue) that is closer to LiDAR ground truth (orange).
• Figure 2: Illustration of monostatic sensing in a mmWave communication system. The mmWave device (left) simultaneously transmits and receives OFDM communication signal for sensing.
• Figure 3: Illustration of angular estimation on a mmWave device.
• Figure 4: Examples of generated radio point clouds, where $\mathbf{S}_{>a} = \{s \!\!\in\!\! \mathbf{S} \, | \, s \!\!>\!\! a \}$. Threshold $a$ is set to 0.1 and 0.2, and the 3D point clouds are projected onto 2D images for for ease of visualization.
• Figure 5: Overview of the multi-frame 3D RF imaging network. Given multiple radio frames and poses, a shared encoder predicts per-frame feature maps and confidence logits. We then warp all features to a reference frame and fuse them into a unified representation. A coarse-to-fine 3D decoder with voxel and depth heads outputs the reconstructed voxel grid and depth map.