Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhanced Cooperative Perception for Autonomous Vehicles Using Imperfect Communication

Ahmad Sarlak, Hazim Alzorgan, Sayed Pedram Haeri Boroujeni, Abolfazl Razi, Rahul Amin

TL;DR

A novel approach to realize an optimized CP under constrained communications by recruiting the best helper from the available list of front vehicles to augment the visual range and enhance the Object Detection (OD) accuracy of the ego vehicle.

Abstract

Sharing and joint processing of camera feeds and sensor measurements, known as Cooperative Perception (CP), has emerged as a new technique to achieve higher perception qualities. CP can enhance the safety of Autonomous Vehicles (AVs) where their individual visual perception quality is compromised by adverse weather conditions (haze as foggy weather), low illumination, winding roads, and crowded traffic. To cover the limitations of former methods, in this paper, we propose a novel approach to realize an optimized CP under constrained communications. At the core of our approach is recruiting the best helper from the available list of front vehicles to augment the visual range and enhance the Object Detection (OD) accuracy of the ego vehicle. In this two-step process, we first select the helper vehicles that contribute the most to CP based on their visual range and lowest motion blur. Next, we implement a radio block optimization among the candidate vehicles to further improve communication efficiency. We specifically focus on pedestrian detection as an exemplary scenario. To validate our approach, we used the CARLA simulator to create a dataset of annotated videos for different driving scenarios where pedestrian detection is challenging for an AV with compromised vision. Our results demonstrate the efficacy of our two-step optimization process in improving the overall performance of cooperative perception in challenging scenarios, substantially improving driving safety under adverse conditions. Finally, we note that the networking assumptions are adopted from LTE Release 14 Mode 4 side-link communication, commonly used for Vehicle-to-Vehicle (V2V) communication. Nonetheless, our method is flexible and applicable to arbitrary V2V communications.

Enhanced Cooperative Perception for Autonomous Vehicles Using Imperfect Communication

TL;DR

A novel approach to realize an optimized CP under constrained communications by recruiting the best helper from the available list of front vehicles to augment the visual range and enhance the Object Detection (OD) accuracy of the ego vehicle.

Abstract

Sharing and joint processing of camera feeds and sensor measurements, known as Cooperative Perception (CP), has emerged as a new technique to achieve higher perception qualities. CP can enhance the safety of Autonomous Vehicles (AVs) where their individual visual perception quality is compromised by adverse weather conditions (haze as foggy weather), low illumination, winding roads, and crowded traffic. To cover the limitations of former methods, in this paper, we propose a novel approach to realize an optimized CP under constrained communications. At the core of our approach is recruiting the best helper from the available list of front vehicles to augment the visual range and enhance the Object Detection (OD) accuracy of the ego vehicle. In this two-step process, we first select the helper vehicles that contribute the most to CP based on their visual range and lowest motion blur. Next, we implement a radio block optimization among the candidate vehicles to further improve communication efficiency. We specifically focus on pedestrian detection as an exemplary scenario. To validate our approach, we used the CARLA simulator to create a dataset of annotated videos for different driving scenarios where pedestrian detection is challenging for an AV with compromised vision. Our results demonstrate the efficacy of our two-step optimization process in improving the overall performance of cooperative perception in challenging scenarios, substantially improving driving safety under adverse conditions. Finally, we note that the networking assumptions are adopted from LTE Release 14 Mode 4 side-link communication, commonly used for Vehicle-to-Vehicle (V2V) communication. Nonetheless, our method is flexible and applicable to arbitrary V2V communications.
Paper Structure (13 sections, 15 equations, 9 figures, 2 tables)

This paper contains 13 sections, 15 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. CAVs with Imperfect Communications
  4. 3D Object Detection
  5. Fusion Methods
  6. System Model
  7. Approximate Vision Range
  8. Motion Blur Level
  9. Communications Parameters
  10. Optimization Problem
  11. Fusion
  12. Simulation
  13. Conclusion

Figures (9)

  • Figure 1: Three fusion approaches used in cooperative perception, including (a) early fusion, (b) late fusion, and (c) integrative analysis.
  • Figure 2: Two scenarios that may occur on the road are: (a) in foggy weather, helper vehicles closer to pedestrians (Green and Purple) can share their cameras with the ego vehicle (Red vehicle), and (b) on a curved road, a helper can detect an accident and share its camera with the ego vehicle.+
  • Figure 3: The individual visual range for vehicles. Recruiting helpers extend the visual range of the ego vehicle through cooperative perception.
  • Figure 4: Objects detection by YOLOv8 to locate the pedestrian. Left: ego vehicle's view, Right: helper vehicle's view. The green box is the ground truth and the red box is the predicted bounding box by YOLOv8.
  • Figure 5: Data set generated with CARLA simulator
  • ...and 4 more figures