Communication-Efficient Collaborative Perception via Information Filling with Codebook

TL;DR

CodeFilling tackles the fundamental trade-off between perception performance and communication cost in collaborative 3D perception by introducing a task-driven codebook-based message representation and an information-filling-driven message selection strategy. By exchanging compact integer codes and coordinating information demands across agents, it avoids redundant data transmission while preserving detection accuracy. The approach supports both homogeneous and heterogeneous collaborations and achieves state-of-the-art perception-communication trade-offs on DAIR-V2X and OPV2VH+ with substantial bandwidth reductions and competitive latency. The work advances practical multi-agent perception by enabling scalable, robust collaboration under diverse sensor configurations and bandwidth constraints.

Abstract

Collaborative perception empowers each agent to improve its perceptual ability through the exchange of perceptual messages with other agents. It inherently results in a fundamental trade-off between perception ability and communication cost. To address this bottleneck issue, our core idea is to optimize the collaborative messages from two key aspects: representation and selection. The proposed codebook-based message representation enables the transmission of integer codes, rather than high-dimensional feature maps. The proposed information-filling-driven message selection optimizes local messages to collectively fill each agent's information demand, preventing information overflow among multiple agents. By integrating these two designs, we propose CodeFilling, a novel communication-efficient collaborative perception system, which significantly advances the perception-communication trade-off and is inclusive to both homogeneous and heterogeneous collaboration settings. We evaluate CodeFilling in both a real-world dataset, DAIR-V2X, and a new simulation dataset, OPV2VH+. Results show that CodeFilling outperforms previous SOTA Where2comm on DAIR-V2X/OPV2VH+ with 1,333/1,206 times lower communication volume. Our code is available at https://github.com/PhyllisH/CodeFilling.

Figures (11)

• Figure 1: CodeFilling avoids redundant messages and achieves more complete detections by transmitting more critical perceptual information with the compact code index message.
• Figure 2: CodeFilling is a novel communication-efficient collaborative 3D detection system. The proposed information-filling-driven message selection and codebook-based message representation contribute to optimizing collaborative messages.
• Figure 3: The information-filling-driven message selection fulfills the information demand with non-redundant information.
• Figure 4: The codebook-based message representation depicts the original feature vector with the most relevant codes.
• Figure 5: In DAIR-V2X, CodeFilling achieves the best perception-communication trade-off in homogeneous & heterogeneous settings.