Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FeaKM: Robust Collaborative Perception under Noisy Pose Conditions

Jiuwu Hao, Liguo Sun, Ti Xiang, Yuting Wan, Haolin Song, Pin Lv

TL;DR

FeaKM tackles the challenge of robust collaborative perception under severe pose uncertainty by introducing a pose-rectification mechanism based on feature-level keypoints matching. The method identifies salient points via a confidence map, computes descriptors, and uses self- and cross-attention with singular value decomposition to derive a fine-grained transformation that aligns partner features before fusion. A multiscale feature fusion strategy further mitigates residual misalignment, leading to improved 3D object detection performance on DAIR-V2X across noisy conditions. This approach enables reliable inter-agent collaboration without relying on precise initial poses, enhancing robustness in real-world urban sensing scenarios.

Abstract

Collaborative perception is essential for networks of agents with limited sensing capabilities, enabling them to work together by exchanging information to achieve a robust and comprehensive understanding of their environment. However, localization inaccuracies often lead to significant spatial message displacement, which undermines the effectiveness of these collaborative efforts. To tackle this challenge, we introduce FeaKM, a novel method that employs Feature-level Keypoints Matching to effectively correct pose discrepancies among collaborating agents. Our approach begins by utilizing a confidence map to identify and extract salient points from intermediate feature representations, allowing for the computation of their descriptors. This step ensures that the system can focus on the most relevant information, enhancing the matching process. We then implement a target-matching strategy that generates an assignment matrix, correlating the keypoints identified by different agents. This is critical for establishing accurate correspondences, which are essential for effective collaboration. Finally, we employ a fine-grained transformation matrix to synchronize the features of all agents and ascertain their relative statuses, ensuring coherent communication among them. Our experimental results demonstrate that FeaKM significantly outperforms existing methods on the DAIR-V2X dataset, confirming its robustness even under severe noise conditions. The code and implementation details are available at https://github.com/uestchjw/FeaKM.

FeaKM: Robust Collaborative Perception under Noisy Pose Conditions

TL;DR

FeaKM tackles the challenge of robust collaborative perception under severe pose uncertainty by introducing a pose-rectification mechanism based on feature-level keypoints matching. The method identifies salient points via a confidence map, computes descriptors, and uses self- and cross-attention with singular value decomposition to derive a fine-grained transformation that aligns partner features before fusion. A multiscale feature fusion strategy further mitigates residual misalignment, leading to improved 3D object detection performance on DAIR-V2X across noisy conditions. This approach enables reliable inter-agent collaboration without relying on precise initial poses, enhancing robustness in real-world urban sensing scenarios.

Abstract

Collaborative perception is essential for networks of agents with limited sensing capabilities, enabling them to work together by exchanging information to achieve a robust and comprehensive understanding of their environment. However, localization inaccuracies often lead to significant spatial message displacement, which undermines the effectiveness of these collaborative efforts. To tackle this challenge, we introduce FeaKM, a novel method that employs Feature-level Keypoints Matching to effectively correct pose discrepancies among collaborating agents. Our approach begins by utilizing a confidence map to identify and extract salient points from intermediate feature representations, allowing for the computation of their descriptors. This step ensures that the system can focus on the most relevant information, enhancing the matching process. We then implement a target-matching strategy that generates an assignment matrix, correlating the keypoints identified by different agents. This is critical for establishing accurate correspondences, which are essential for effective collaboration. Finally, we employ a fine-grained transformation matrix to synchronize the features of all agents and ascertain their relative statuses, ensuring coherent communication among them. Our experimental results demonstrate that FeaKM significantly outperforms existing methods on the DAIR-V2X dataset, confirming its robustness even under severe noise conditions. The code and implementation details are available at https://github.com/uestchjw/FeaKM.

Paper Structure

This paper contains 17 sections, 7 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. RELATED WORKS
  3. Collaborative Perception
  4. Pose Rectification
  5. PROBLEM STATEMENT
  6. Our method
  7. Overall Architecture
  8. Pose Rectification Module
  9. Interest Point Extraction
  10. Target Matching
  11. Multiscale Feature Fusion
  12. experiment
  13. Dataset and Experiment Settings
  14. Quantitative Results
  15. Qualitative Results
  16. ...and 2 more sections

Figures (3)

  • Figure 1: Overall architecture of proposed FeaKM.
  • Figure 2: Detection results on DAIR-V2X dataset with different noise errors. Our proposed method outperforms other methods.
  • Figure 3: Visualization of perception results on DAIR-V2X dataset. Green boxes represent ground-truth while red boxes are prediction. Our method achieves more accurate detection.