Vision-Aided Channel Prediction Based on Image Segmentation at Street Intersection Scenarios

TL;DR

This work addresses the challenge of predicting vehicular channel characteristics for V2I without increasing spectral overhead by leveraging out-of-band visual data. It proposes a vision-aided framework that uses YOLOv8-based instance segmentation to isolate the Rx vehicle in RGB frames captured at the base station, feeding segmented images into a regression network (ResNet-34) to predict $PL$, the Rice $K$-factor, and RMS delay spread $\tau_{rms}$, validated on datasets from two urban street intersections. The approach demonstrates that single segmented images provide superior prediction accuracy and robust generalization to new streets and unseen vehicles, outperforming original and fully segmented image inputs. This method offers a practical means to enhance intelligent vehicular communications and 6G-enabled V2I systems by exploiting visual context without consuming additional spectrum, with strong potential for real-time channel awareness and optimization.

Abstract

Intelligent vehicular communication with vehicle road collaboration capability is a key technology enabled by 6G, and the integration of various visual sensors on vehicles and infrastructures plays a crucial role. Moreover, accurate channel prediction is foundational to realizing intelligent vehicular communication. Traditional methods are still limited by the inability to balance accuracy and operability based on substantial spectrum resource consumption and highly refined description of environment. Therefore, leveraging out-of-band information introduced by visual sensors provides a new solution and is increasingly applied across various communication tasks. In this paper, we propose a computer vision (CV)-based prediction model for vehicular communications, realizing accurate channel characterization prediction including path loss, Rice K-factor and delay spread based on image segmentation. First, we conduct extensive vehicle-to-infrastructure measurement campaigns, collecting channel and visual data from various street intersection scenarios. The image-channel dataset is generated after a series of data post-processing steps. Image data consists of individual segmentation of target user using YOLOv8 network. Subsequently, established dataset is used to train and test prediction network ResNet-32, where segmented images serve as input of network, and various channel characteristics are treated as labels or target outputs of network. Finally, self-validation and cross-validation experiments are performed. The results indicate that models trained with segmented images achieve high prediction accuracy and remarkable generalization performance across different streets and target users. The model proposed in this paper offers novel solutions for achieving intelligent channel prediction in vehicular communications.

Figures (9)

• Figure 1: Framework of proposed vision-based channel prediction model.
• Figure 2: Measurement scenarios for different streets.
• Figure 3: Examples of measurement-based channel characteristics. (a) PL; (b) RMS delay spread; (c) Rice K-factor; (d) APDP.
• Figure 4: Workflow diagram of image segmentation network and channel prediction network.
• Figure 5: Examples of three types of images on Streets A and B.