Synesthesia of Machines (SoM)-Aided Online FDD Precoding via Heterogeneous Multi-Modal Sensing: A Vertical Federated Learning Approach

TL;DR

The paper tackles pilot overhead and latency in FDD precoding for multi-vehicle networks by introducing a vertical federated learning framework that fuses heterogeneous multi-modal sensing (GPS, RGB, LiDAR) at the RSU. It develops three dedicated preprocessing streams, an offline H-MVMM training pipeline, and a label-free online updating mechanism via the PCSI-Simulator to update precoding weights without true CSI labels. The approach achieves near-WMMSE performance with perfect CSI while substantially reducing pilot requirements and enabling dynamic adaptation to changing users and sensor configurations; it also demonstrates robustness to sensing imperfections and favorable data-efficiency versus centralized learning. The work offers practical implications for latency-sensitive vehicular networks and outlines avenues for handling channel dynamics and scalable user grouping in future systems.

Abstract

This paper investigates a heterogeneous multi-vehicle, multi-modal sensing (H-MVMM) aided online precoding problem. The proposed H-MVMM scheme utilizes a vertical federated learning (VFL) framework to minimize pilot sequence length and optimize the sum rate. This offers a promising solution for reducing latency in frequency division duplexing systems. To achieve this, three preprocessing modules are designed to transform raw sensory data into informative representations relevant to precoding. The approach effectively addresses local data heterogeneity arising from diverse on-board sensor configurations through a well-structured VFL training procedure. Additionally, a label-free online model updating strategy is introduced, enabling the H-MVMM scheme to adapt its weights flexibly. This strategy features a pseudo downlink channel state information label simulator (PCSI-Simulator), which is trained using a semi-supervised learning (SSL) approach alongside an online loss function. Numerical results show that the proposed method can closely approximate the performance of traditional optimization techniques with perfect channel state information, achieving a significant 90.6\% reduction in pilot sequence length.

Figures (13)

• Figure 1: An illustration of the system model.
• Figure 2: Processing flow of raw multi-modal sensory data.
• Figure 3: Acquisition process of the indicator vector.
• Figure 4: Lightweight processing of LiDAR point cloud.
• Figure 5: The architecture of PCSI-Simulator.