Environment-Aware Channel Inference via Cross-Modal Flow: From Multimodal Sensing to Wireless Channels

TL;DR

The paper tackles the challenge of obtaining full CSI in dynamic mmWave MIMO without pilots by leveraging multimodal environmental sensing. It introduces a cross-modal flow matching framework that fuses camera, LiDAR, and GPS data into a latent channel representation and learns a velocity field to morph this latent into the wireless channel distribution, enabling real-time CSI inference. Key contributions include a tractable conditional flow matching formulation with a modality-alignment loss, a multimodal stochastic encoder, and a neural velocity field, validated on a Sionna-Blender data generator with substantial NMSE and spectral efficiency improvements over pilot-based and sensing-based baselines. The work demonstrates significant practical impact by reducing pilot overhead and delivering robust, low-latency CSI for downstream beamforming in dynamic NG networks.

Abstract

Accurate channel state information (CSI) underpins reliable and efficient wireless communication. However, acquiring CSI via pilot estimation incurs substantial overhead, especially in massive multiple-input multiple-output (MIMO) systems operating in high-Doppler environments. By leveraging the growing availability of environmental sensing data, this treatise investigates pilot-free channel inference that estimates complete CSI directly from multimodal observations, including camera images, LiDAR point clouds, and GPS coordinates. In contrast to prior studies that rely on predefined channel models, we develop a data-driven framework that formulates the sensing-to-channel mapping as a cross-modal flow matching problem. The framework fuses multimodal features into a latent distribution within the channel domain, and learns a velocity field that continuously transforms the latent distribution toward the channel distribution. To make this formulation tractable and efficient, we reformulate the problem as an equivalent conditional flow matching objective and incorporate a modality alignment loss, while adopting low-latency inference mechanisms to enable real-time CSI estimation. In experiments, we build a procedural data generator based on Sionna and Blender to support realistic modeling of sensing scenes and wireless propagation. System-level evaluations demonstrate significant improvements over pilot- and sensing-based benchmarks in both channel estimation accuracy and spectral efficiency for the downstream beamforming task.

Figures (13)

• Figure 1: Physical scenario where the base station is equipped with a camera, a LiDAR system and a server, and the user is deployed with a GPS. At the server, the complete CSI between the base station and the user is estimated from multimodal sensing data, including image, point cloud and coordinate.
• Figure 2: Transmission protocol with frame-based CSI acquisition, where the environment-aware channel inference is employed for CSI update.
• Figure 3: Network structure of the multimodal stochastic encoder.
• Figure 4: Network structure of the neural velocity field.
• Figure 5: Inference pipeline that enables the cross-modality evolution from multimodal sensing data to wireless channel representation.