Large Wireless Model (LWM): A Foundation Model for Wireless Channels

TL;DR

Large Wireless Model's ability to learn from large-scale wireless data opens a promising direction for intelligent systems that can efficiently adapt to diverse tasks with limited data, paving the way for addressing key challenges in wireless communication and sensing systems.

Abstract

This paper presents Large Wireless Model (LWM) -- the world's first foundation model for wireless channels. Designed as a task-agnostic model, LWM generates universal, rich, contextualized channel embeddings (features) that potentially enhance performance across a wide range of downstream tasks in wireless communication and sensing systems. Towards this objective, LWM, which has a transformer-based architecture, was pre-trained in a self-supervised manner on large-scale wireless channel datasets. Our results show consistent improvements in downstream tasks when using the LWM embeddings compared to raw channel representations, especially in scenarios with high-complexity machine learning tasks and limited training datasets. This LWM's ability to learn from large-scale wireless data opens a promising direction for intelligent systems that can efficiently adapt to diverse tasks with limited data, paving the way for addressing key challenges in wireless communication and sensing systems.

Figures (6)

• Figure 1: This figure depicts the offline pre-training and online embedding generation process for LWM. The channel is divided into fixed-size patches, which are linearly embedded and combined with positional encodings before being passed through a Transformer encoder. During self-supervised pre-training, some embeddings are masked, and LWM leverages self-attention to extract deep features, allowing the decoder to reconstruct the masked values. For downstream tasks, the generated LWM embeddings enhance performance. The right block shows the LWM architecture, inspired by vaswani2023attentionneed.
• Figure 2: Multi-head Attention Mechanism
• Figure 3: This figure compares beam prediction F1-score performance between raw channels and their inferred LWM embeddings, based on a total of 10388 training raw channels, and highlights their relative effectiveness.
• Figure 4: This figure compares F1-scores for LoS/NLoS classification using models trained on raw wireless channels and LWM embeddings across different percentages of the $6639$ training samples. CLS embeddings are $32 \times$ smaller than raw channels, while channel embeddings are $4 \times$ larger. CLS embeddings inferred from noisy (imperfect) raw channels are also included to demonstrate LWM’s robustness to noise.
• Figure 5: This figure visualizes the distribution of users in the DeepMIMO Denver scenario based on their LoS/NLoS status (top row) and strongest DFT beam index among $8$ beams (bottom row). User channels are projected into $2$D using t-SNE, comparing raw channels, task-agnostic (general-purpose) LWM embeddings, and fine-tuned LWM embeddings for each task. As seen in Fig. \ref{['fig:los']}, CLS embeddings clearly separate LoS and NLoS channels, enabling high zero-shot classification and strong initialization for downstream training with minimal data. Fine-tuning further enhances downstream task performance.