A Multi-Modal Simulation Framework to Enable Digital Twin-based V2X Communications in Dynamic Environments

TL;DR

This work tackles the challenge of reliable V2X communications in dynamic urban environments at mmWave/sub-THz frequencies by proposing a Digital Twin–driven, multi-modal simulation framework. The framework combines CARLA-based multi-sensor simulation, Unreal Engine rendering, shape-grammar–driven 3D reconstruction, ITU-materialEM properties, and Remcom Wireless InSite ray-tracing to produce synchronized sensor and channel data, enabling realistic DTs of vehicular scenarios. A key demonstration is the DT-aided blockage handover, where DT-informed beam subsets and real-time channel data yield faster and more robust link restoration compared to conventional 5G NR gradient-search approaches, especially as codebook size and antenna dimensions grow. The framework provides realistic benchmarks and ML-ready datasets for decision-making tasks such as beam prediction, channel estimation, and VE positioning, with broad implications for reliable V2X operations in next-generation networks.

Abstract

Digital Twins (DTs) for physical wireless environments have been recently proposed as accurate virtual representations of the propagation environment that can enable multi-layer decisions at the physical communication equipment. At high-frequency bands, DTs can help to overcome the challenges emerging in high mobility conditions featuring vehicular environments. In this paper, we propose a novel data-driven workflow for the creation of the DT of a Vehicle-to-Everything (V2X) communication scenario and a multi-modal simulation framework for the generation of realistic sensor data and accurate mmWave/sub-THz wireless channels. The proposed method leverages an automotive simulation and testing framework and an accurate ray-tracing channel simulator. Simulations over an urban scenario show the achievable realistic sensor and channel modelling both at the infrastructure and at ego-vehicles. We showcase the proposed framework on the DT-aided blockage handover task for V2X link restoration, leveraging the framework's dynamic channel generation capabilities for realistic vehicular blockage simulation.

Figures (6)

• Figure 1: Interaction between a real-world V2X network and its high-fidelity digitized representation. The urban environment and the vehicles' movements govern wireless propagation in the real world. The digital replica uses high-definition reconstructed 3D maps, realistic sensor simulations, and accurate 3D ray tracing to control the communication equipment.
• Figure 2: Geometric variation of the most powerful ray-tracing channel paths between an RSU and a VE operating at 28 GHz in an urban environment when affected (blue) or not affected (orange) by vehicular blockage from a truck in terms of (a) geometrical configuration of the main paths in the environment, and (b) directions of arrival and delays of the simulated channel paths.
• Figure 3: DT construction workflow. Acquisitions from multiple sensors at the vehicle or the communication infrastructure are preprocessed by sensor fusion and semantic instance segmentation to determine the components of the 3D scene and their materials. Procedural digital reconstruction is performed and utilized for multi-modal sensor simulation within an Unreal Engine-based automotive framework and for accurate wireless propagation simulation through a ray-tracing simulator.
• Figure 4: Simulation of RGB camera, LiDAR and semantic segmentation camera sensors and ray-tracing channel at a simulation frame for the selected urban scenario both at the communication infrastructure and at an ego-vehicle.
• Figure 5: Beamforming gain ratio versus BS antenna array dimensions between DT-aided blockage handover and the 5G NR approach for codebook cardinality $K = 2$.