Environment Semantic Communication: Enabling Distributed Sensing Aided Networks

TL;DR

The paper addresses the high beam training overhead in mmWave/THz communication by introducing distributed sensing nodes that capture RGB imagery and extract environment semantics to aid beam prediction. It proposes a three-stage pipeline: (1) extract environment semantics (bounding boxes and masks) at distributed nodes using YOLOv7, (2) identify and track the transmitter at the basestation with receive-power and semantic cues, and (3) predict the optimal beam using single-instance or sequence-based models (FCNN, LeNet, and LSTM). The authors formalize the system model, formulate the beam-prediction problem, and demonstrate (via the DeepSense 6G dataset) that semantic data from distributed nodes can achieve top-3 beam accuracy exceeding 75%, while substantially reducing data burdens. The work shows the practical impact of distributed sensing and semantic communication for scalable, low-latency beam management in dynamic V2I scenarios and sets a foundation for proactive, context-aware wireless systems.

Abstract

Millimeter-wave (mmWave) and terahertz (THz) communication systems require large antenna arrays and use narrow directive beams to ensure sufficient receive signal power. However, selecting the optimal beams for these large antenna arrays incurs a significant beam training overhead, making it challenging to support applications involving high mobility. In recent years, machine learning (ML) solutions have shown promising results in reducing the beam training overhead by utilizing various sensing modalities such as GPS position and RGB images. However, the existing approaches are mainly limited to scenarios with only a single object of interest present in the wireless environment and focus only on co-located sensing, where all the sensors are installed at the communication terminal. This brings key challenges such as the limited sensing coverage compared to the coverage of the communication system and the difficulty in handling non-line-of-sight scenarios. To overcome these limitations, our paper proposes the deployment of multiple distributed sensing nodes, each equipped with an RGB camera. These nodes focus on extracting environmental semantics from the captured RGB images. The semantic data, rather than the raw images, are then transmitted to the basestation. This strategy significantly alleviates the overhead associated with the data storage and transmission of the raw images. Furthermore, semantic communication enhances the system's adaptability and responsiveness to dynamic environments, allowing for prioritization and transmission of contextually relevant information. Experimental results on the DeepSense 6G dataset demonstrate the effectiveness of the proposed solution in reducing the sensing data transmission overhead while accurately predicting the optimal beams in realistic communication environments.

Figures (15)

• Figure 1: The figure shows the overall system model of the proposed setup. The distributed nodes extract environment semantic information from the RGB images, which is subsequently transmitted to the basestation. This semantic information is then utilized for beam prediction at the basestation.
• Figure 2: The figure illustrates the selection process of the ULA, the sub-region, and the corresponding distributed node.
• Figure 3: This figure outlines the different stages of the proposed solution. First, we extract environment semantics from the raw RGB images, transmitting them to the basestation. In the second stage, we identify the transmitter in the initial frame and track it over the subsequent frames. The final step involves using this semantic information of the transmitter, accumulated in the second stage, for beam prediction.
• Figure 4: The figure illustrates the environment semantics extraction stage in our proposed solution. In particular, a camera installed at the distributed node captures real-time images of the wireless environment, which a machine learning model then processes to extract the bounding boxes and masks of the mobile objects present in the images.
• Figure 5: The figure shows the transmitter identification and object association-based tracking module. The transmitter is identified in the first frame using the receive power vector and then tracked for the remaining frames using the nearest neighbor algorithm.