Multi-Modality Sensing in mmWave Beamforming for Connected Vehicles Using Deep Learning

TL;DR

This paper tackles the overhead of beam alignment in mmWave vehicular communications by proposing a deep learning solution that leverages multi-modal sensing (position, vision, and LiDAR) to predict a subset of candidate beams (top-$M$). The authors design a multimodal neural network with three unimodal feature extractors, train it on real-world DeepSense 6G data, and demonstrate high top-$M$ accuracies (up to $98.19\%$ for top-$13$) and substantial overhead reductions. They show that the approach can be integrated with 5G-NR beamforming workflows, reducing beam sweeping time from tens of milliseconds to the sub-millisecond range, while preserving high received-power ratios. The work highlights the practical impact of out-of-band information fusion for proactive, low-latency beam management in both V2I and V2V scenarios, paving the way for more robust mmWave V2X systems.

Abstract

Beamforming techniques are considered as essential parts to compensate severe path losses in millimeter-wave (mmWave) communications. In particular, these techniques adopt large antenna arrays and formulate narrow beams to obtain satisfactory received powers. However, performing accurate beam alignment over narrow beams for efficient link configuration by traditional standard defined beam selection approaches, which mainly rely on channel state information and beam sweeping through exhaustive searching, imposes computational and communications overheads. And, such resulting overheads limit their potential use in vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) communications involving highly dynamic scenarios. In comparison, utilizing out-of-band contextual information, such as sensing data obtained from sensor devices, provides a better alternative to reduce overheads. This paper presents a deep learning-based solution for utilizing the multi-modality sensing data for predicting the optimal beams having sufficient mmWave received powers so that the best V2I and V2V line-of-sight links can be ensured proactively. The proposed solution has been tested on real-world measured mmWave sensing and communication data, and the results show that it can achieve up to 98.19% accuracies while predicting top-13 beams. Correspondingly, when compared to existing been sweeping approach, the beam sweeping searching space and time overheads are greatly shortened roughly by 79.67% and 91.89%, respectively which confirm a promising solution for beamforming in mmWave enabled communications.

Figures (7)

• Figure 1: Illustration of our considered mmWave enabled V2I and V2V communications system model.
• Figure 2: The details diagram of the proposed deep learning model on multi-modality sensing for mmWave beamforming, which is mainly composed of three feature extraction and one top-$M$ beam selection components.
• Figure 3: Visual representation of receiver vehicle’s GPS location data points (100 and 400 samples for V2I and V2V scenarios, respectively) along with corresponding best beam indices out of 64 beams on Google Map satellite view.
• Figure 4: The results of loss and accuracies while training the proposed model, indicating how the learning performances of the model on training data from all considered scenarios are improving over $40$ number of epochs.
• Figure 5: The performances of average achieved top-$M$ accuracies and received power ratios in percentages tested on the vehicle-to-infrastructure and vehicle-to-vehicle scenarios.