Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perception

TL;DR

This survey addresses collaborative perception for vehicle-to-everything autonomous driving by detailing CP architectures, datasets, and taxonomy; it analyzes performance, latency compensation, robustness to noise, and sim-to-real generalization through extensive experiments. It highlights the strengths and trade-offs of early, intermediate, and late fusion, and proposes future directions to improve bandwidth efficiency, simulator realism, sensor placement, and cross-domain generalization. The work provides practical guidance for researchers and industry, including a public codebase for reproducible experimentation and an agenda for addressing privacy and security in V2X CP systems.

Abstract

Vehicle-to-everything (V2X) autonomous driving opens up a promising direction for developing a new generation of intelligent transportation systems. Collaborative perception (CP) as an essential component to achieve V2X can overcome the inherent limitations of individual perception, including occlusion and long-range perception. In this survey, we provide a comprehensive review of CP methods for V2X scenarios, bringing a profound and in-depth understanding to the community. Specifically, we first introduce the architecture and workflow of typical V2X systems, which affords a broader perspective to understand the entire V2X system and the role of CP within it. Then, we thoroughly summarize and analyze existing V2X perception datasets and CP methods. Particularly, we introduce numerous CP methods from various crucial perspectives, including collaboration stages, roadside sensors placement, latency compensation, performance-bandwidth trade-off, attack/defense, pose alignment, etc. Moreover, we conduct extensive experimental analyses to compare and examine current CP methods, revealing some essential and unexplored insights. Specifically, we analyze the performance changes of different methods under different bandwidths, providing a deep insight into the performance-bandwidth trade-off issue. Also, we examine methods under different LiDAR ranges. To study the model robustness, we further investigate the effects of various simulated real-world noises on the performance of different CP methods, covering communication latency, lossy communication, localization errors, and mixed noises. In addition, we look into the sim-to-real generalization ability of existing CP methods. At last, we thoroughly discuss issues and challenges, highlighting promising directions for future efforts. Our codes for experimental analysis will be public at https://github.com/memberRE/Collaborative-Perception.

Figures (9)

• Figure 1: A diagram illustrating V2X scenarios. The red car faces the occlusion issue, and the green car faces the long-range perception issue. By obtaining extra perceptual information from other vehicles (V2V) or infrastructure (V2I), these vehicles can achieve a holistic perception of their surroundings, improving traffic safety.
• Figure 2: Overview of this survey.
• Figure 3: The illustration of a typical V2X system architecture in practical applications.
• Figure 4: Illustration of different collaboration stage.
• Figure 5: Performance-Bandwidth Trade-off. The x-axis represents the Bandwidth in log2 scale, and the y-axis represents the AP performance.