Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-agent Collaborative Perception for Robotic Fleet: A Systematic Review

Apoorv Singh, Gaurav Raut, Alka Choudhary

TL;DR

The paper surveys collaborative perception in multi-robot fleets, addressing how shared sensing extends visibility beyond individual sensors. It organizes a taxonomy of collaboration stages, problem-driven methods, datasets, and evaluation frameworks across autonomous vehicles, AMRs, drones, and surveillance contexts. A notable finding is a demonstration of substantial performance gains from collaboration, exemplified by an improvement of $>200\%$ when using 10+ agents, and it reviews benchmarking datasets and simulators used for 3D object detection and segmentation. The work highlights challenges in communication, privacy, and model heterogeneity, and outlines future directions including foundational multi-modal models, scalable data management for large fleets, and privacy-preserving data sharing.

Abstract

Collaborative perception in multi-robot fleets is a way to incorporate the power of unity in robotic fleets. Collaborative perception refers to the collective ability of multiple entities or agents to share and integrate their sensory information for a more comprehensive understanding of their environment. In other words, it involves the collaboration and fusion of data from various sensors or sources to enhance perception and decision-making capabilities. By combining data from diverse sources, such as cameras, lidar, radar, or other sensors, the system can create a more accurate and robust representation of the environment. In this review paper, we have summarized findings from 20+ research papers on collaborative perception. Moreover, we discuss testing and evaluation frameworks commonly accepted in academia and industry for autonomous vehicles and autonomous mobile robots. Our experiments with the trivial perception module show an improvement of over 200% with collaborative perception compared to individual robot perception. Here's our GitHub repository that shows the benefits of collaborative perception: https://github.com/synapsemobility/synapseBEV

Multi-agent Collaborative Perception for Robotic Fleet: A Systematic Review

TL;DR

The paper surveys collaborative perception in multi-robot fleets, addressing how shared sensing extends visibility beyond individual sensors. It organizes a taxonomy of collaboration stages, problem-driven methods, datasets, and evaluation frameworks across autonomous vehicles, AMRs, drones, and surveillance contexts. A notable finding is a demonstration of substantial performance gains from collaboration, exemplified by an improvement of when using 10+ agents, and it reviews benchmarking datasets and simulators used for 3D object detection and segmentation. The work highlights challenges in communication, privacy, and model heterogeneity, and outlines future directions including foundational multi-modal models, scalable data management for large fleets, and privacy-preserving data sharing.

Abstract

Collaborative perception in multi-robot fleets is a way to incorporate the power of unity in robotic fleets. Collaborative perception refers to the collective ability of multiple entities or agents to share and integrate their sensory information for a more comprehensive understanding of their environment. In other words, it involves the collaboration and fusion of data from various sensors or sources to enhance perception and decision-making capabilities. By combining data from diverse sources, such as cameras, lidar, radar, or other sensors, the system can create a more accurate and robust representation of the environment. In this review paper, we have summarized findings from 20+ research papers on collaborative perception. Moreover, we discuss testing and evaluation frameworks commonly accepted in academia and industry for autonomous vehicles and autonomous mobile robots. Our experiments with the trivial perception module show an improvement of over 200% with collaborative perception compared to individual robot perception. Here's our GitHub repository that shows the benefits of collaborative perception: https://github.com/synapsemobility/synapseBEV
Paper Structure (24 sections, 2 equations, 4 figures, 3 tables)

This paper contains 24 sections, 2 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Collaborative Perception
  3. Stages of Collaboration
  4. Why do we need Collaboration Perception?
  5. Problem Driven Collaboration Methods
  6. High Communication Bandwidth
  7. Agent Localization Error
  8. Latency/ Communication Issues
  9. Model/ Task Discrepancies
  10. Privacy/ Cybersecurity
  11. Evaluation metrics for vision generative models
  12. Evaluation Methods
  13. Testing Platforms
  14. Datasets
  15. Simulators
  16. ...and 9 more sections

Figures (4)

  • Figure 1: A warehouse use case of collaborative perception. Using collaborative perception, two critical perception challenges, (1) Occlusions and (2) Long-range detections, can be resolved.
  • Figure 2: Visualization of collaborative perception. Orange cars (Ego-car) won't have any blind spots once other smart cars in white and smart infrastructure in black share data about the potential collision with the jaywalking pedestrian.
  • Figure 3: Different stages of collaboration perception, depending on how much information is processed in the perception network before sharing information.
  • Figure 4: Synpase synapsebev visualization. [Left]: Individual perception data. [Right]: Collaborative perception data. Key: Blue: Ego robot; Red: Other robots; Gray-scale: Visibility (Black is completely invisible and white is completely visible).