SCA-LLM: Spectral-Attentive LLM-Based Wireless World Modeling for Agentic Communications

TL;DR

This work tackles the challenge of building scalable wireless world models for agentic communications by predicting multi-step CSI trajectories. It introduces SCA-LLM, a spectral-attentive adapter that bridges CSI to a pre-trained LLM (GPT-2) using a 2-D DCT-based multi-spectral channel attention module, enabling parameter-efficient adaptation while keeping the LLM backbone largely frozen. Empirical results on QuaDRiGa-based MIMO-OFDM data show state-of-the-art NMSE performance and strong zero-shot generalization to unseen environments, with significant gains over prior LLM-based approaches. Ablation studies confirm the critical role of the SCA adapter and the LLM backbone in achieving robust, cross-scenario channel prediction.

Abstract

Future AI-native wireless networks are moving from reactive optimization to agentic decision-making that can sense, predict, and plan under fast-varying channels. This calls for wireless world models that can predict and roll out channel dynamics, for which multi-step channel state information (CSI) prediction offers a practical short-horizon look-ahead. Recent advances in foundation sequence models further motivate large language models (LLMs) as general-purpose dynamics learners when suitably adapted to non-text time-series signals. However, bridging CSI to LLMs is non-trivial because an effective adapter must expose informative spectral and temporal evolution patterns, while prior designs provide limited inductive bias to capture such channel structures. To this end, we propose SCA-LLM, a spectral-attentive LLM-based wireless world modeling framework that bridges CSI to LLMs via a spectral-channel attention (SCA) adapter. Specifically, the SCA adapter performs multi-spectral representation learning to extract informative channel features and align CSI with the LLM's sequence modeling capability, enabling parameter-efficient adaptation while keeping the LLM backbone largely frozen. Extensive simulations show that SCA-LLM achieves state-of-the-art prediction performance and strong zero-shot generalization, yielding up to -2.4 dB normalized mean squared error (NMSE) advantage over the previous LLM based method. Our ablation studies further confirm the effectiveness of the proposed SCA adapter in mitigating domain mismatch.

Figures (8)

• Figure 1: The architecture of the proposed SCA-LLM framework. The adapter component encompasses the CSI embedding layer and the spectral channel adaptation (SCA) module.
• Figure 2: NMSE performance versus SNR between UMa and UMi NLOS scenarios with $\text{the UE velocity} = 0~\text{km/h}$.
• Figure 3: NMSE performance versus SNR between UMa and UMi NLOS scenarios with $\text{the UE velocity} = 50~\text{km/h}$.
• Figure 4: NMSE performance versus UE velocity between UMa and UMi NLOS scenarios with $\text{SNR}=10~\text{dB}$.
• Figure 5: NMSE performance versus prediction step $p$ between UMa and UMi NLOS scenarios with $\text{UE velocity}=50~\text{km/h}$ and $\text{SNR}=10~\text{dB}$. The total prediction horizon is $P=6$.