From Raw Data to Shared 3D Semantics: Task-Oriented Communication for Multi-Robot Collaboration

TL;DR

The paper tackles the data–decision bottleneck in multi-robot systems operating in unknown 3D environments by proposing a decentralized task-oriented semantic communication framework. It constructs a geometry-aware perception pipeline using PiDiNet to extract sparse structural edges, 3D anchors, and object primitives, transmitting only event-driven semantic updates to build a lightweight shared 3D semantic scene for decentralized coordination. Key contributions include the semantic message format with explicit bit budgeting, an event-driven transmission policy with priority rules, and a decentralized pipeline for semantic-aware task allocation, motion planning, and cooperative transport, all without centralized fusion or dense maps. Experiments show dramatic bandwidth reductions (over 200×) and substantial improvements in task completion speed (up to ~4×) and travel efficiency (≈3×) as team size scales, demonstrating the practical impact of aligning communication with task utility in bandwidth-constrained multi-robot collaboration.

Abstract

Multi-robot systems (MRS) rely on exchanging raw sensory data to cooperate in complex three-dimensional (3D) environments. However, this strategy often leads to severe communication congestion and high transmission latency, significantly degrading collaboration efficiency. This paper proposes a decentralized task-oriented semantic communication framework for multi-robot collaboration in unknown 3D environments. Each robot locally extracts compact, task-relevant semantics using a lightweight Pixel Difference Network (PiDiNet) with geometric processing. It shares only these semantic updates to build a task-sufficient 3D scene representation that supports cooperative perception, navigation, and object transport. Our numerical results show that the proposed method exhibits a dramatic reduction in communication overhead from $858.6$ Mb to $4.0$ Mb (over $200\times$ compression gain) while improving collaboration efficiency by shortening task completion from $1,054$ to $281$ steps.

Figures (5)

• Figure 1: Overview of the task-oriented semantic communication framework for multi-robot collaboration. First, multi-modal local sensing data (RGB, point clouds, and odometry) is processed to extract compact structural, geometric, and object semantics, effectively filtering out task-irrelevant information. Second, an event-driven semantic communication mechanism transmits these lightweight semantic packets. Finally, the received data is aggregated into a shared 3D semantic scene, visualized as sparse edge maps, which serves as a lightweight representation to directly drive semantic-aware decision-making and cooperative task execution.
• Figure 2: The performance comparison between raw and semantic communication under bandwidth constraints: (a) communication cost, (b) task completion steps, and (c) total traveled distance.
• Figure 3: A visualization of robot trajectories under semantic communication (left) and raw communication (right).
• Figure 4: The cumulative number of discovered and delivered targets.
• Figure 5: An illustration of the shared 3D semantic scene evolving in the four-robot semantic setting, from initialization to the final state. It incrementally constructs the shared task-oriented semantic scene enabled by event-driven semantic communication.