Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HeatV2X: Scalable Heterogeneous Collaborative Perception via Efficient Alignment and Interaction

Yueran Zhao, Zhang Zhang, Chao Sun, Tianze Wang, Chao Yue, Nuoran Li

TL;DR

HeatV2X tackles multi-modal, multi-agent heterogeneity and scalability in V2X collaborative perception. It introduces Local Heterogeneous Fine-Tuning (LHFT) with Hetero-Aware Adapters and Global Collaborative Fine-Tuning (GCFT) with a Multi-Cognitive Adapter to align heterogeneous agents and strengthen cross-agent interaction while keeping training costs low. The approach trains a strong base agent end-to-end and then rapidly adapts new agents with minimal parameter updates, enabling scalable expansion. Empirical results on OPV2V-H and DAIR-V2X show superior perception performance and reduced training overhead compared to state-of-the-art methods, with robustness under noise and latency. The work offers practical significance for deploying scalable V2X perception systems.

Abstract

Vehicle-to-Everything (V2X) collaborative perception extends sensing beyond single vehicle limits through transmission. However, as more agents participate, existing frameworks face two key challenges: (1) the participating agents are inherently multi-modal and heterogeneous, and (2) the collaborative framework must be scalable to accommodate new agents. The former requires effective cross-agent feature alignment to mitigate heterogeneity loss, while the latter renders full-parameter training impractical, highlighting the importance of scalable adaptation. To address these issues, we propose Heterogeneous Adaptation (HeatV2X), a scalable collaborative framework. We first train a high-performance agent based on heterogeneous graph attention as the foundation for collaborative learning. Then, we design Local Heterogeneous Fine-Tuning and Global Collaborative Fine-Tuning to achieve effective alignment and interaction among heterogeneous agents. The former efficiently extracts modality-specific differences using Hetero-Aware Adapters, while the latter employs the Multi-Cognitive Adapter to enhance cross-agent collaboration and fully exploit the fusion potential. These designs enable substantial performance improvement of the collaborative framework with minimal training cost. We evaluate our approach on the OPV2V-H and DAIR-V2X datasets. Experimental results demonstrate that our method achieves superior perception performance with significantly reduced training overhead, outperforming existing state-of-the-art approaches. Our implementation will be released soon.

HeatV2X: Scalable Heterogeneous Collaborative Perception via Efficient Alignment and Interaction

TL;DR

HeatV2X tackles multi-modal, multi-agent heterogeneity and scalability in V2X collaborative perception. It introduces Local Heterogeneous Fine-Tuning (LHFT) with Hetero-Aware Adapters and Global Collaborative Fine-Tuning (GCFT) with a Multi-Cognitive Adapter to align heterogeneous agents and strengthen cross-agent interaction while keeping training costs low. The approach trains a strong base agent end-to-end and then rapidly adapts new agents with minimal parameter updates, enabling scalable expansion. Empirical results on OPV2V-H and DAIR-V2X show superior perception performance and reduced training overhead compared to state-of-the-art methods, with robustness under noise and latency. The work offers practical significance for deploying scalable V2X perception systems.

Abstract

Vehicle-to-Everything (V2X) collaborative perception extends sensing beyond single vehicle limits through transmission. However, as more agents participate, existing frameworks face two key challenges: (1) the participating agents are inherently multi-modal and heterogeneous, and (2) the collaborative framework must be scalable to accommodate new agents. The former requires effective cross-agent feature alignment to mitigate heterogeneity loss, while the latter renders full-parameter training impractical, highlighting the importance of scalable adaptation. To address these issues, we propose Heterogeneous Adaptation (HeatV2X), a scalable collaborative framework. We first train a high-performance agent based on heterogeneous graph attention as the foundation for collaborative learning. Then, we design Local Heterogeneous Fine-Tuning and Global Collaborative Fine-Tuning to achieve effective alignment and interaction among heterogeneous agents. The former efficiently extracts modality-specific differences using Hetero-Aware Adapters, while the latter employs the Multi-Cognitive Adapter to enhance cross-agent collaboration and fully exploit the fusion potential. These designs enable substantial performance improvement of the collaborative framework with minimal training cost. We evaluate our approach on the OPV2V-H and DAIR-V2X datasets. Experimental results demonstrate that our method achieves superior perception performance with significantly reduced training overhead, outperforming existing state-of-the-art approaches. Our implementation will be released soon.

Paper Structure

This paper contains 25 sections, 6 equations, 15 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. V2X Heterogeneous Perception
  4. Multi-Agent Heterogeneous
  5. Multi-Modal Heterogeneous
  6. Fine-Tuning Strategy
  7. Parameter-Efficient Fine-Tuning
  8. Collaborative Perception Based on Fine-Tuning
  9. Methodology
  10. Collaborative Base Model
  11. Collaborative Aligning and Fine-Tuning
  12. Hetero-Aware Adapter (HA Adapter)
  13. Local Heterogeneous Fine-Tuning (LHFT)
  14. Multi-Cognitive Adapter (MC Adapter)
  15. Global Collaborative Fine-Tuning (GCFT)
  16. ...and 10 more sections

Figures (15)

  • Figure 1: Comparison with heterogeneous collaborative perception methods on OPV2V-H. (a) Typical heterogeneous collaboration scenarios and expansion processes. (b) Comparison of heterogeneous methods in terms of perception performance and model efficiency with 4 agents. (c) and (d) The trainable parameters and training GPU time versus the number of heterogeneous agents. Please note that the pre-training of each modality is also included in the calculation.
  • Figure 2: Overview of HeatV2X. Stage 1: We obtain the collaborative base model via full-parameter training. Stage 2: The new agent freezes most of the weights shared from the base and performs rapid alignment using the Hetero-Aware Adapters (HA Adapter). Stage 3: The collaboration conducts cross-agent interaction based on the Multi-Cognitive Adapter (MC Adapter). Stage 4: Collaborative perception.
  • Figure 3: The structure of adapters. (a): The Hetero-Aware Vanilla Adapter. (b): The MC Adapter.
  • Figure 4: HEAL - Scene 1
  • Figure 5: HEAL - Scene 2
  • ...and 10 more figures