CoPEFT: Fast Adaptation Framework for Multi-Agent Collaborative Perception with Parameter-Efficient Fine-Tuning

TL;DR

This work tackles the problem of distribution-shift robustness in multi-agent collaborative perception by presenting CoPEFT, a lightweight, parameter-efficient fine-tuning framework. It introduces two complementary components—the macro-level Collaboration Adapter and the micro-level Agent Prompt—to adapt a pre-trained collaborative perception model to new deployment environments while updating less than 1% of parameters. Through extensive experiments on OPV2V, DAIR-V2X, and V2XSet, CoPEFT demonstrates substantial performance gains over scratch training, standard PEFT methods, and domain adaptation baselines, particularly at low data availability. The approach offers a plug-and-play solution that preserves general collaboration knowledge and injects environment-specific cues, enabling practical deployment in resource-constrained settings with reduced training costs and data requirements.

Abstract

Multi-agent collaborative perception is expected to significantly improve perception performance by overcoming the limitations of single-agent perception through exchanging complementary information. However, training a robust collaborative perception model requires collecting sufficient training data that covers all possible collaboration scenarios, which is impractical due to intolerable deployment costs. Hence, the trained model is not robust against new traffic scenarios with inconsistent data distribution and fundamentally restricts its real-world applicability. Further, existing methods, such as domain adaptation, have mitigated this issue by exposing the deployment data during the training stage but incur a high training cost, which is infeasible for resource-constrained agents. In this paper, we propose a Parameter-Efficient Fine-Tuning-based lightweight framework, CoPEFT, for fast adapting a trained collaborative perception model to new deployment environments under low-cost conditions. CoPEFT develops a Collaboration Adapter and Agent Prompt to perform macro-level and micro-level adaptations separately. Specifically, the Collaboration Adapter utilizes the inherent knowledge from training data and limited deployment data to adapt the feature map to new data distribution. The Agent Prompt further enhances the Collaboration Adapter by inserting fine-grained contextual information about the environment. Extensive experiments demonstrate that our CoPEFT surpasses existing methods with less than 1\% trainable parameters, proving the effectiveness and efficiency of our proposed method.

Figures (4)

• Figure 1: Illustration of CoPEFT. We mitigate the impact of inconsistent data distribution on collaborative perception by dynamically combining general knowledge derived from the training data with specific and environmental knowledge obtained from the deployment data. Here, general knowledge encompasses the general patterns of collaborative perception, specific knowledge represents the disparities between the new deployment and the training data, and environmental knowledge refers to fine-grained contextual information. Consequently, CoPEFT can fast adapt a well-trained model to various deployment environments at a low cost.
• Figure 2: The overall architecture of CoPEFT. It involves standard components in intermediate collaboration augmented with two lightweight elements: a Collaboration Adapter and an Agent Prompt. (a) The Collaboration Adapter, guided by several collaborative perception priors, adapts the feature maps from a macro-level perspective for new data. (b) The Agent Prompt offers fine-grained environmental information from a micro-level perspective, which can be conceptualized as the insertion of a virtual agent to further assist in adapting feature maps. By updating only the parameters of the Collaboration Adapter, Agent Prompt, and Decoder Network, CoPEFT effectively realizes the dynamic combination of general, specific, and environmental knowledge for fast adaptation.
• Figure 3: Qualitative comparison. The green and red 3D bounding boxes represent ground truth and prediction, respectively. Best viewed in color.
• Figure 4: Comparisons of CoPEFT variations.