Resource-Efficient Beam Prediction in mmWave Communications with Multimodal Realistic Simulation Framework

TL;DR

The paper tackles the challenge of rapid, resource-efficient mmWave beam prediction in dynamic urban environments. It introduces a multimodal realistic simulation framework that combines CARLA for sensing data with MATLAB for mmWave channel modeling, and proposes Cross-modal Relational Knowledge Distillation (CRKD) to transfer knowledge from a multimodal teacher to a radar-only student. Through relational and beam-space distillation, the radar-only model achieves near-teacher performance while using a fraction of the parameters, enabling sub-millisecond inference on edge devices. These results demonstrate the practicality of sensor-balanced beam prediction under tight computational budgets and highlight avenues for domain-robustness and broader modality integration.

Abstract

Beamforming is a key technology in millimeter-wave (mmWave) communications that improves signal transmission by optimizing directionality and intensity. However, conventional channel estimation methods, such as pilot signals or beam sweeping, often fail to adapt to rapidly changing communication environments. To address this limitation, multimodal sensing-aided beam prediction has gained significant attention, using various sensing data from devices such as LiDAR, radar, GPS, and RGB images to predict user locations or network conditions. Despite its promising potential, the adoption of multimodal sensing-aided beam prediction is hindered by high computational complexity, high costs, and limited datasets. Thus, in this paper, a novel resource-efficient learning framework is introduced for beam prediction, which leverages a custom-designed cross-modal relational knowledge distillation (CRKD) algorithm specifically tailored for beam prediction tasks, to transfer knowledge from a multimodal network to a radar-only student model, achieving high accuracy with reduced computational cost. To enable multimodal learning with realistic data, a novel multimodal simulation framework is developed while integrating sensor data generated from the autonomous driving simulator CARLA with MATLAB-based mmWave channel modeling, and reflecting real-world conditions. The proposed CRKD achieves its objective by distilling relational information across different feature spaces, which enhances beam prediction performance without relying on expensive sensor data. Simulation results demonstrate that CRKD efficiently distills multimodal knowledge, allowing a radar-only model to achieve $94.62%$ of the teacher performance. In particular, this is achieved with just $10%$ of the teacher network's parameters, thereby significantly reducing computational complexity and dependence on multimodal sensor data.

Figures (9)

• Figure 1: Model training in a virtual environment that combines multiple simulators.
• Figure 2: System model for multi-beam prediction with cross-modal knowledge distillation.
• Figure 3: Multimodal realistic simulation framework based on autonomous driving tool CARLA and MATLAB.
• Figure 4: The proposed structure of cross-modal knowledge distillation from multimodal (LiDAR, RGB, radar, and GPS) to monomodal (radar).
• Figure 5: Relational Knowledge Distillation with different relationships.