SynthRM: A Synthetic Data Platform for Vision-Aided Mobile System Simulation

TL;DR

SynthRM introduces a physically well-posed vision-aided wireless sensing framework by aligning radio propagation with the surfaces visible to an ego-centric camera through the Visible-Aligned-Surface (VAS) paradigm. It combines city-scale procedural generation with a depth-back-projected, mesh-centric ray-tracing backend (Sionna-RT) to produce a pixel-aligned radio-textured mesh, enabling dense, geometry-driven learning of channel behavior. The dataset provides synchronized RGB-D, geometry, material, and per-pixel Path Gain and SINR on the visible surfaces, supporting rigorous evaluation and generalization tests against traditional top-down or loosely-coupled multimodal datasets. This open, scalable platform enables robust pre-training and sim-to-real transfer for next-generation mobile systems that rely on environment-aware sensing and communication, including V2X, UAVs, and XR scenarios.

Abstract

Vision-aided wireless sensing is emerging as a cornerstone of 6G mobile computing. While data-driven approaches have advanced rapidly, establishing a precise geometric correspondence between ego-centric visual data and radio propagation remains a challenge. Existing paradigms typically either associate 2D topology maps and auxiliary information with radio maps, or provide 3D perspective views limited by sparse radio data. This spatial representation flattens the complex vertical interactions such as occlusion and diffraction that govern signal behavior in urban environments, rendering the task of cross-view signal inference mathematically ill-posed. To resolve this geometric ambiguity, we introduce SynthRM, a scalable synthetic data platform. SynthRM implements a Visible-Aligned-Surface simulation strategy: rather than probing a global volumetric grid, it performs ray-tracing directly onto the geometry exposed to the sensor. This approach ensures pixel-level consistency between visual semantics and electromagnetic response, transforming the learning objective into a physically well-posed problem. We demonstrate the platform's capabilities by presenting a diverse, city-scale dataset derived from procedurally generated environments. By combining efficient procedural synthesis with high-fidelity electromagnetic modeling, SynthRM provides a transparent, accessible foundation for developing next-generation mobile systems for environment-aware sensing and communication.

Figures (11)

• Figure 1: The SynthRM Data Collection Workflow. The pipeline transitions from procedural scene generation (Composition) to frustum-specific mesh recovery (Alignment), culminating in surface-bound ray tracing (Simulation). This ensures radio metrics are topologically mapped to the visible surfaces.
• Figure 2: Procedural Generation of Urban Environments. This figure illustrates the two branches of the output from the PCG process. (a) shows the PCG output; (b) shows the 3D geometry (polygons) used for radio simulation; (c) shows the photorealistic rendering in UE5.
• Figure 3: The reconstructed surface for a specific perspective camera overlayed on the 3D city scene model. Decimation of vertices has been performed during rendering to clearly demonstrate the 3D structure.
• Figure 4: Radiation patterns for the three transmitter antenna configurations used in SynthRM. The SISO model exhibits uniform radiation, while the MIMO arrays demonstrate directional beamforming characteristics.
• Figure 5: Multi-Modal Data Sample. Every frame includes pixel-aligned channels: (a) RGB Image, (b) Depth Map, (c) Surface Normals, (d) Roughness, (e) Diffuse Material properties, and (f) The Radio Map projected onto the VAS.