CoDS: Enhancing Collaborative Perception in Heterogeneous Scenarios via Domain Separation

TL;DR

CoDS tackles feature discrepancies in heterogeneous collaborative perception by introducing domain separation with LSCR and DADS, complemented by the Domain Alignment Mutual Information (DAMI) loss. The fully convolutional adapter preserves efficiency while aligning neighbor features to the ego domain without re-training encoders, using encoder-specific and encoder-agnostic modules to separate domain-dependent from task-related information; DAMI maximizes cross-view mutual information to reinforce robust alignment, expressed as $ ext{I}_{ ext{DAMI}}=rac{1}{N_{ ext{nei}}}\sum_{j=1}^{N_{ ext{nei}}} ext{I}( ilde{f}_i; ilde{f}_{j ightarrow i})$. Extensive experiments on V2V4Real, OPV2V, and V2XSet across various numbers of neighbors and under pose noise demonstrate that CoDS consistently improves detection accuracy while delivering high inference speed, achieving a favorable trade-off between performance and efficiency. The work confirms the practicality of domain separation for heterogeneous multi-agent perception and offers a scalable, lightweight alternative to transformer-based adapters.

Abstract

Collaborative perception has been proven to improve individual perception in autonomous driving through multi-agent interaction. Nevertheless, most methods often assume identical encoders for all agents, which does not hold true when these models are deployed in real-world applications. To realize collaborative perception in actual heterogeneous scenarios, existing methods usually align neighbor features to those of the ego vehicle, which is vulnerable to noise from domain gaps and thus fails to address feature discrepancies effectively. Moreover, they adopt transformer-based modules for domain adaptation, which causes the model inference inefficiency on mobile devices. To tackle these issues, we propose CoDS, a Collaborative perception method that leverages Domain Separation to address feature discrepancies in heterogeneous scenarios. The CoDS employs two feature alignment modules, i.e., Lightweight Spatial-Channel Resizer (LSCR) and Distribution Alignment via Domain Separation (DADS). Besides, it utilizes the Domain Alignment Mutual Information (DAMI) loss to ensure effective feature alignment. Specifically, the LSCR aligns the neighbor feature across spatial and channel dimensions using a lightweight convolutional layer. Subsequently, the DADS mitigates feature distribution discrepancy with encoder-specific and encoder-agnostic domain separation modules. The former removes domain-dependent information and the latter captures task-related information. During training, the DAMI loss maximizes the mutual information between aligned heterogeneous features to enhance the domain separation process. The CoDS employs a fully convolutional architecture, which ensures high inference efficiency. Extensive experiments demonstrate that the CoDS effectively mitigates feature discrepancies in heterogeneous scenarios and achieves a trade-off between detection accuracy and inference efficiency.

Figures (10)

• Figure 1: Comparison between homogeneous and heterogeneous scenarios. In homogeneous scenario, Ego vehicle (No. 1) and CAV (No. 2) employ the identical model, resulting in shared homogeneous features. In heterogeneous scenario, Ego vehicle (No. 1) and CAV (No. 3) or RSU (No. 4) employ distinct models, leading to shared heterogeneous features.
• Figure 2: Different frameworks for heterogeneous collaboration. (a) Re-trains encoders for neighbor agents, (b) designs a dedicated heterogeneous feature fusion module, and (c) incorporates a plug-and-play adapter that requires only fine-tuning based on the existing homogeneous feature fusion module.
• Figure 3: Overview of CoDS. The ego and neighbor agents first extract features using distinct encoders. After receiving heterogeneous neighbor features, the ego applies the Lightweight Spatial-Channel Resizer (LSCR) and Distribution Alignment via Domain Separation (DADS) to align these features. During training, Domain Alignment Mutual Information (DAMI) loss is used to ensure effective feature alignment.
• Figure 4: The architecture of the DADS. The encoder-specific domain separation modules remove private information tied to individual encoders, whereas the encoder-agnostic modules capture shared task-related information.
• Figure 5: Illustration of the DAMI loss. (a) We maximize mutual information between aligned features by bringing views of the same scene closer together and pushing views of different scenes further apart, which is achieved through contrastive learning on positive and negative pairs. (b) and (c) illustrate the construction of positive and negative samples. In scene $b_1$, the $j$-th aligned neighbor feature $\widetilde{f}^{b_1}_{j\rightarrow i}$ serves as a positive sample for the aligned ego feature $\widetilde{f}^{b_1}_{i}$. (b) If the ego in another scene $b_2$ also has the $j$-th agent, the negative sample is the aligned neighbor feature $\widetilde{f}^{b_2}_{j\rightarrow i}$ from scene $b_2$. (c) Otherwise, the negative sample is the aligned ego feature $\widetilde{f}^{b_2}_{i}$ from scene $b_2$.