A Unified Cloud-Edge-Terminal Framework for Multimodal Integrated Sensing and Communication

Abstract

The transition to 6G calls for tightly integrated sensing and communication to support mission-critical services such as autonomous driving, embodied AI, and high-precision telemedicine. However, most existing ISAC designs rely on a single sensing modality (often RF), which limits environmental understanding and becomes a bottleneck in complex and dynamic scenes. This motivates a shift from single-modal to multimodal ISAC, where heterogeneous sensors (e.g., radar, LiDAR, and cameras) complement each other to improve robustness and semantic awareness. In this article, we first summarize key challenges for multimodal ISAC, including heterogeneous fusion, communication overhead, and scalable system design. We then highlight three enabling technologies: large AI models, semantic communications, and multi-agent systems, and discuss how their combination can enable task-oriented multimodal perception. Building on these insights, we propose a unified cloud-edge-terminal (CET) framework that hierarchically distributes intelligence and supports three adaptive operation modes: global fusion mode (GFM), cooperative relay mode (CRM), and peer interaction mode (PIM). A case study evaluates the framework across three modes, demonstrating that GFM achieves the highest accuracy, PIM minimizes latency, and CRM strikes an optimal balance between performance and efficiency. Finally, we conclude with open research issues and future directions.

Figures (4)

• Figure 1: The illustration of three technologies. (a) Large AI models. (b) Semantic communication. (c) Multi-agent system.
• Figure 2: The illustration of the proposed CET framework for multimodal ISAC.
• Figure 3: The illustration of the three modes in the CET framework.(a) GFM, (b) CRM, and (c) PIM.
• Figure 4: The beam prediction accuracy under different schemes in (a) Scenario 31 and (b) Scenario 33.