Extended Visibility of Autonomous Vehicles via Optimized Cooperative Perception under Imperfect Communication

TL;DR

The paper tackles extending autonomous-vehicle perception via cooperative perception when V2V links are imperfect. It introduces a two-stage optimization: first, select M helpers to maximize a fractional objective $G(s)/D(s)$ that fuses location, visual range, and motion blur; second, allocate NR-V2X resources to maximize the CP throughput per energy, solved with Dinkelbach's algorithm, Frank–Wolfe, and SDP relaxations. Key contributions include a dynamic helper recruitment framework that accounts for spatial positions and channel quality, an integrated resource-allocation pipeline, and a late-fusion CP implementation using YOLOv8 validated on CARLA data, achieving about a 10% improvement in pedestrian detection. This work demonstrates that CP with optimized helper selection and resource management can significantly enhance AV safety and reliability in complex, communication-constrained environments.

Abstract

Autonomous Vehicles (AVs) rely on individual perception systems to navigate safely. However, these systems face significant challenges in adverse weather conditions, complex road geometries, and dense traffic scenarios. Cooperative Perception (CP) has emerged as a promising approach to extending the perception quality of AVs by jointly processing shared camera feeds and sensor readings across multiple vehicles. This work presents a novel CP framework designed to optimize vehicle selection and networking resource utilization under imperfect communications. Our optimized CP formation considers critical factors such as the helper vehicles' spatial position, visual range, motion blur, and available communication budgets. Furthermore, our resource optimization module allocates communication channels while adjusting power levels to maximize data flow efficiency between the ego and helper vehicles, considering realistic models of modern vehicular communication systems, such as LTE and 5G NR-V2X. We validate our approach through extensive experiments on pedestrian detection in challenging scenarios, using synthetic data generated by the CARLA simulator. The results demonstrate that our method significantly improves upon the perception quality of individual AVs with about 10% gain in detection accuracy. This substantial gain uncovers the unleashed potential of CP to enhance AV safety and performance in complex situations.

Figures (13)

• Figure 1: Three fusion approaches used in CP, including (a) early fusion, (b) late fusion, and (c) integrative analysis.
• Figure 2: Two scenarios where CP helps: (a) in foggy weather, the ego vehicle (blue) has poor perception quality, and two of the helper vehicles (green and red) share their camera feeds for improved pedestrian detection via CP, (b) on a winding road, the helper vehicle (truck) helps the ego vehicle (police car) detect the otherwise invisible accident.
• Figure 3: The workflow of multi-vehicle CP implementation.
• Figure 4: C-V2X subchannelization and example of utilization resources. Resource contain of Transport Block (TB) and SCI.
• Figure 5: The individual visual range for vehicles. Recruiting helpers extend the visual range of the ego vehicle through CP.