Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

V2X-DG: Domain Generalization for Vehicle-to-Everything Cooperative Perception

Baolu Li, Zongzhe Xu, Jinlong Li, Xinyu Liu, Jianwu Fang, Xiaopeng Li, Hongkai Yu

TL;DR

This paper is the first work to study the Domain Generalization problem of LiDAR-based V2X cooperative perception (V2X-DG) for 3D detection based on four widely-used open source datasets: OPV2V, V2XSet, V2V4Real and DAIR-V2X.

Abstract

LiDAR-based Vehicle-to-Everything (V2X) cooperative perception has demonstrated its impact on the safety and effectiveness of autonomous driving. Since current cooperative perception algorithms are trained and tested on the same dataset, the generalization ability of cooperative perception systems remains underexplored. This paper is the first work to study the Domain Generalization problem of LiDAR-based V2X cooperative perception (V2X-DG) for 3D detection based on four widely-used open source datasets: OPV2V, V2XSet, V2V4Real and DAIR-V2X. Our research seeks to sustain high performance not only within the source domain but also across other unseen domains, achieved solely through training on source domain. To this end, we propose Cooperative Mixup Augmentation based Generalization (CMAG) to improve the model generalization capability by simulating the unseen cooperation, which is designed compactly for the domain gaps in cooperative perception. Furthermore, we propose a constraint for the regularization of the robust generalized feature representation learning: Cooperation Feature Consistency (CFC), which aligns the intermediately fused features of the generalized cooperation by CMAG and the early fused features of the original cooperation in source domain. Extensive experiments demonstrate that our approach achieves significant performance gains when generalizing to other unseen datasets while it also maintains strong performance on the source dataset.

V2X-DG: Domain Generalization for Vehicle-to-Everything Cooperative Perception

TL;DR

This paper is the first work to study the Domain Generalization problem of LiDAR-based V2X cooperative perception (V2X-DG) for 3D detection based on four widely-used open source datasets: OPV2V, V2XSet, V2V4Real and DAIR-V2X.

Abstract

LiDAR-based Vehicle-to-Everything (V2X) cooperative perception has demonstrated its impact on the safety and effectiveness of autonomous driving. Since current cooperative perception algorithms are trained and tested on the same dataset, the generalization ability of cooperative perception systems remains underexplored. This paper is the first work to study the Domain Generalization problem of LiDAR-based V2X cooperative perception (V2X-DG) for 3D detection based on four widely-used open source datasets: OPV2V, V2XSet, V2V4Real and DAIR-V2X. Our research seeks to sustain high performance not only within the source domain but also across other unseen domains, achieved solely through training on source domain. To this end, we propose Cooperative Mixup Augmentation based Generalization (CMAG) to improve the model generalization capability by simulating the unseen cooperation, which is designed compactly for the domain gaps in cooperative perception. Furthermore, we propose a constraint for the regularization of the robust generalized feature representation learning: Cooperation Feature Consistency (CFC), which aligns the intermediately fused features of the generalized cooperation by CMAG and the early fused features of the original cooperation in source domain. Extensive experiments demonstrate that our approach achieves significant performance gains when generalizing to other unseen datasets while it also maintains strong performance on the source dataset.

Paper Structure

This paper contains 15 sections, 7 equations, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Overview of V2X-DG Architecture
  5. Cooperative Mixup Augmentation based Generalization
  6. Cooperation Feature Consistency
  7. Overall Loss Function
  8. Experiments
  9. Datasets
  10. Experimental Setup
  11. Comparison Methods.
  12. Quantitative Comparisons.
  13. Ablation Study.
  14. Qualitative Comparisons.
  15. Conclusions

Figures (3)

  • Figure 1: Illustration of the domain gap (Agent Number Distribution Gap, Cooperative Agent Type Gap) for V2X cooperative perception between source domain to unseen domains. Best viewed in color.
  • Figure 2: Overview V2X-DG framework. CMAG bridges the Agent Number Distribution Gap and Cooperative Agent Type Gap by three components: Mixup Agent, Point Augmentation, and Probabilistic Gate. $\mathcal{L}_{CFC}$ is the regularization for robust generalized feature representation learning. V2X-DG is only used in the source-domain training. Best view in color.
  • Figure 3: Qualitative comparison on OPV2V2ALL setting.Green and red 3D bounding boxes represent the ground truth and prediction respectively. The detection errors are highlighted using purple arrows. OPV2V is used as source domain here.