Bridging Visual and Wireless Sensing: A Unified Radiation Field for 3D Radio Map Construction

TL;DR

This work tackles the challenge of building high-fidelity 3D radio maps by bridging visual perception and wireless propagation. It introduces URF-GS, a unified radio–optical radiation-field representation based on 3D Gaussian splatting and physics‑informed inverse rendering to jointly recover geometry, material properties, and signal propagation from multimodal observations. The approach yields strong gains over NeRF-based methods in both accuracy and generalization, demonstrated across indoor 60 GHz scenes, Wi‑Fi AP deployment, and robot path planning, along with immersive VR visualization. By enabling perception, interaction, and communication within a single radiative field, URF-GS lays a foundation for next‑generation wireless networks and digital-twin applications.

Abstract

The emerging applications of next-generation wireless networks (e.g., immersive 3D communication, low-altitude networks, and integrated sensing and communication) necessitate high-fidelity environmental intelligence. 3D radio maps have emerged as a critical tool for this purpose, enabling spectrum-aware planning and environment-aware sensing by bridging the gap between physical environments and electromagnetic signal propagation. However, constructing accurate 3D radio maps requires fine-grained 3D geometric information and a profound understanding of electromagnetic wave propagation. Existing approaches typically treat optical and wireless knowledge as distinct modalities, failing to exploit the fundamental physical principles governing both light and electromagnetic propagation. To bridge this gap, we propose URF-GS, a unified radio-optical radiation field representation framework for accurate and generalizable 3D radio map construction based on 3D Gaussian splatting (3D-GS) and inverse rendering. By fusing visual and wireless sensing observations, URF-GS recovers scene geometry and material properties while accurately predicting radio signal behavior at arbitrary transmitter-receiver (Tx-Rx) configurations. Experimental results demonstrate that URF-GS achieves up to a 24.7% improvement in spatial spectrum prediction accuracy and a 10x increase in sample efficiency for 3D radio map construction compared with neural radiance field (NeRF)-based methods. This work establishes a foundation for next-generation wireless networks by integrating perception, interaction, and communication through holistic radiation field reconstruction.

Figures (11)

• Figure 1: Comparison of constructed 3D radio maps in an indoor environment using URF-GS versus various baselines. (a) Environmental Setting: The upper image presents the room viewed from the visual sensor (i.e., a camera). The bottom image displays the top-down layout of the indoor environment where the Rx can move arbitrarily within this space. Three Tx locations (P1–P3) are evaluated: a large number of samples (of different RX positions) are collected for P1, whereas P2 (10 samples) and P3 (zero-shot) have access to a limited number of samples. (b) Constructed Radio Maps: The 3D radio maps generated at Rx location R1 via different methods, corresponding to the specified Tx locations. URF-GS achieves the highest prediction accuracy among the baselines, closely matching the ground truth in all scenarios (i.e., across locations P1–P3).
• Figure 2: Performance comparison of the URF-GS, RF-3DGS, NeRF2, and WRF-GS+ methods in terms of PSNR, SSIM, and LPIPS metrics over different training dataset sizes.
• Figure 3: Datasets. (a) Bistro. The green and white cubes represent the positions of transmitters and receivers, respectively. (b) Modified Wi3room. The green and white spheres represent the positions of the transmitters and receivers, respectively.
• Figure 4: The average received power of all receivers at different Tx positions.
• Figure 5: A visualization of the GT and predicted Tx positions on the indices $10$ and $23$ in the Bistro scene, where the coordinate projection transformation is used in the projection model.