Large Multimodal Model-Aided Scheduling for 6G Autonomous Communications

TL;DR

This work tackles scheduling in dynamic 6G autonomous networks where rapid channel changes undermine traditional QoS guarantees. It proposes LMM-PS, a framework that uses large multimodal models to predict future channel parameters from visual sensing and pilot data, enabling preemptive, channel-aware scheduling via a PF-based mechanism. Core components include GLIPv2-based object detection for localization, LLaVA-based mobility prediction, and Transformer-based NLoS parameter estimation, integrated into a two-stage prediction and scheduling pipeline. Empirical results show substantial throughput improvements (e.g., $>32\%$ over PF and $>11\%$ over DRL-based methods) and robust performance under blockages and mobility, illustrating the practical potential of LMM-enabled autonomous communications.

Abstract

Recently, large language models (LLMs) have gained significant attention for their ability to generate fast and accurate answer to the given query. These models have evolved into large multimodal models (LMMs), which can interpret and analyze multimodal inputs such as images and text. With the exponential growth of AI functionalities in autonomous devices, the central unit (CU), a digital processing unit performing AI inference, needs to handle LMMs to effectively control these devices. To ensure seamless command delivery to devices, the CU must perform the scheduling, which involves resource block (RB) allocation for data transmission and modulation and coding scheme (MCS) index selection based on the channel conditions. This task is challenging in many practical environments in 6G, where even small user movement can cause abrupt channel changes. In this paper, we propose a novel LMM-based scheduling technique to address this challenge. Our key idea is to leverage LMM to predict future channel parameters (e.g., distance, angles, and path gain) by analyzing the visual sensing information as well as pilot signals. By exploiting LMMs to predict the presence of reliable path and geometric information of users from the visual sensing information, and then combining these with past channel states from pilot signals, we can accurately predict future channel parameters. Using these predictions, we can preemptively make channel-aware scheduling decisions. From the numerical evaluations, we show that the proposed technique achieves more than 30% throughput gain over the conventional scheduling techniques.

Figures (14)

• Figure 1: LLM/LMM-based task-oriented communication system.
• Figure 2: Conceptual illustration of the proposed LMM-PS.
• Figure 3: Illustration of the conventional CQI feedback-based scheduling. When the BS makes scheduling decisions based on the outdated CQI feedback, they may no longer reflect the true channel condition, leading to degradation of beamforming gain and transmission rate.
• Figure 4: Overall process of the proposed LMM-PS.
• Figure 5: Basic structure of LMM.