Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BERT4MIMO: A Foundation Model using BERT Architecture for Massive MIMO Channel State Information Prediction

Ferhat Ozgur Catak, Murat Kuzlu, Umit Cali

TL;DR

This work introduces BERT4MIMO, a foundation-model approach that adapts BERT-style transformers to the reconstruction of high-dimensional CSI in massive MIMO for NextG networks. By engineering temporal and feature embeddings and a 12-layer, 12-head Transformer encoder, the model learns robust CSI representations under stationary, high-mobility, and urban macro scenarios and supports masked-input prediction with a mean-squared error objective. Key contributions include a MATLAB-generated CSI dataset across 3 scenarios, an architecture with explicit real/imaginary CSI handling, and extensive evaluations showing strong reconstruction performance (MSE ≈ 0.011) and substantial gains over baselines (≈30×). The findings demonstrate the practical potential of a foundation model for CSI across diverse wireless environments, enabling improved beamforming, sensing, and network optimization in NextG systems.

Abstract

Massive MIMO (Multiple-Input Multiple-Output) is an advanced wireless communication technology, using a large number of antennas to improve the overall performance of the communication system in terms of capacity, spectral, and energy efficiency. The performance of MIMO systems is highly dependent on the quality of channel state information (CSI). Predicting CSI is, therefore, essential for improving communication system performance, particularly in MIMO systems, since it represents key characteristics of a wireless channel, including propagation, fading, scattering, and path loss. This study proposes a foundation model inspired by BERT, called BERT4MIMO, which is specifically designed to process high-dimensional CSI data from massive MIMO systems. BERT4MIMO offers superior performance in reconstructing CSI under varying mobility scenarios and channel conditions through deep learning and attention mechanisms. The experimental results demonstrate the effectiveness of BERT4MIMO in a variety of wireless environments.

BERT4MIMO: A Foundation Model using BERT Architecture for Massive MIMO Channel State Information Prediction

TL;DR

This work introduces BERT4MIMO, a foundation-model approach that adapts BERT-style transformers to the reconstruction of high-dimensional CSI in massive MIMO for NextG networks. By engineering temporal and feature embeddings and a 12-layer, 12-head Transformer encoder, the model learns robust CSI representations under stationary, high-mobility, and urban macro scenarios and supports masked-input prediction with a mean-squared error objective. Key contributions include a MATLAB-generated CSI dataset across 3 scenarios, an architecture with explicit real/imaginary CSI handling, and extensive evaluations showing strong reconstruction performance (MSE ≈ 0.011) and substantial gains over baselines (≈30×). The findings demonstrate the practical potential of a foundation model for CSI across diverse wireless environments, enabling improved beamforming, sensing, and network optimization in NextG systems.

Abstract

Massive MIMO (Multiple-Input Multiple-Output) is an advanced wireless communication technology, using a large number of antennas to improve the overall performance of the communication system in terms of capacity, spectral, and energy efficiency. The performance of MIMO systems is highly dependent on the quality of channel state information (CSI). Predicting CSI is, therefore, essential for improving communication system performance, particularly in MIMO systems, since it represents key characteristics of a wireless channel, including propagation, fading, scattering, and path loss. This study proposes a foundation model inspired by BERT, called BERT4MIMO, which is specifically designed to process high-dimensional CSI data from massive MIMO systems. BERT4MIMO offers superior performance in reconstructing CSI under varying mobility scenarios and channel conditions through deep learning and attention mechanisms. The experimental results demonstrate the effectiveness of BERT4MIMO in a variety of wireless environments.
Paper Structure (35 sections, 31 equations, 6 figures, 5 tables, 1 algorithm)

This paper contains 35 sections, 31 equations, 6 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Massive MIMO Channel State Information
  4. Physical Layer Channel Models
  5. CSI Matrix Preprocessing
  6. Masking
  7. Dataset Details and Simulation Scenarios
  8. Dataset Description
  9. Dataset Generation
  10. Dataset Preprocessing
  11. Scenarios and Their Characteristics
  12. BERT4MIMO Architecture
  13. Input Representation and Preprocessing
  14. Temporal and Feature Embedding Layers
  15. Temporal Embedding
  16. ...and 20 more sections

Figures (6)

  • Figure 1: An illustrative overview of BERT4MIMO's role in modern wireless communication systems.
  • Figure 2: The architecture of BERT4MIMO
  • Figure 3: Effect of masking ratio on reconstruction MSE.
  • Figure 4: Heatmap of Cross-Scenario Performance. Each cell represents the MSE for a combination of training and testing scenarios.
  • Figure 5: Error distribution across all subcarrier groups.
  • ...and 1 more figures