Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CSI-MAE: A Masked Autoencoder-based Channel Foundation Model

Jun Jiang, Xiaolong Ruan, Shugong Xu

TL;DR

The paper addresses the limited generalization and high data/compute costs of existing Channel Foundation Models by introducing CSI-MAE, a masked autoencoder-based CFM trained on standardized 3GPP channel models. It uses a ViT-based encoder to learn universal CSI representations from two-channel input with a high masking ratio and a lightweight decoder for downstream tasks. CSI-MAE demonstrates cross-scenario generalization, strong zero-shot transfer, and competitive or superior performance compared to supervised baselines when fully finetuned, while significantly reducing training costs via a lightweight finetuning strategy. Scaling analyses show data diversity and model capacity improve performance, with clear dimension-dependent effects, highlighting practical benefits for ISAC applications and robust wireless perception across environments.

Abstract

Self-Supervised Learning (SSL) has emerged as a key technique in machine learning, tackling challenges such as limited labeled data, high annotation costs, and variable wireless channel conditions. It is essential for developing Channel Foundation Models (CFMs), which extract latent features from channel state information (CSI) and adapt to different wireless settings. Yet, existing CFMs have notable drawbacks: heavy reliance on scenario-specific data hinders generalization, they focus on single/dual tasks, and lack zero-shot learning ability. In this paper, we propose CSI-MAE, a generalized CFM leveraging masked autoencoder for cross-scenario generalization. Trained on 3GPP channel model datasets, it integrates sensing and communication via CSI perception and generation, proven effective across diverse tasks. A lightweight decoder finetuning strategy cuts training costs while maintaining competitive performance. Under this approach, CSI-MAE matches or surpasses supervised models. With full-parameter finetuning, it achieves the state-of-the-art performance. Its exceptional zero-shot transferability also rivals supervised techniques in cross-scenario applications, driving wireless communication innovation.

CSI-MAE: A Masked Autoencoder-based Channel Foundation Model

TL;DR

The paper addresses the limited generalization and high data/compute costs of existing Channel Foundation Models by introducing CSI-MAE, a masked autoencoder-based CFM trained on standardized 3GPP channel models. It uses a ViT-based encoder to learn universal CSI representations from two-channel input with a high masking ratio and a lightweight decoder for downstream tasks. CSI-MAE demonstrates cross-scenario generalization, strong zero-shot transfer, and competitive or superior performance compared to supervised baselines when fully finetuned, while significantly reducing training costs via a lightweight finetuning strategy. Scaling analyses show data diversity and model capacity improve performance, with clear dimension-dependent effects, highlighting practical benefits for ISAC applications and robust wireless perception across environments.

Abstract

Self-Supervised Learning (SSL) has emerged as a key technique in machine learning, tackling challenges such as limited labeled data, high annotation costs, and variable wireless channel conditions. It is essential for developing Channel Foundation Models (CFMs), which extract latent features from channel state information (CSI) and adapt to different wireless settings. Yet, existing CFMs have notable drawbacks: heavy reliance on scenario-specific data hinders generalization, they focus on single/dual tasks, and lack zero-shot learning ability. In this paper, we propose CSI-MAE, a generalized CFM leveraging masked autoencoder for cross-scenario generalization. Trained on 3GPP channel model datasets, it integrates sensing and communication via CSI perception and generation, proven effective across diverse tasks. A lightweight decoder finetuning strategy cuts training costs while maintaining competitive performance. Under this approach, CSI-MAE matches or surpasses supervised models. With full-parameter finetuning, it achieves the state-of-the-art performance. Its exceptional zero-shot transferability also rivals supervised techniques in cross-scenario applications, driving wireless communication innovation.
Paper Structure (10 sections, 5 equations, 6 figures, 5 tables)

This paper contains 10 sections, 5 equations, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Proposed Framework
  3. Model Architecture
  4. Downstream Tasks
  5. Experiments
  6. Dataset
  7. Results analysis
  8. Zero-shot Performance
  9. Scaling Law
  10. Conclustion

Figures (6)

  • Figure 1: The overall framework for CFMs.
  • Figure 2: Our proposed CSI-MAE architecture.
  • Figure 3: Input for antenna-domain and subcarrier-domain extrapolation.
  • Figure 4: The chart compares performance across scenarios and antenna/subcarrier-domain extrapolations.
  • Figure 5: Performance comparison of CSI-MAE for the positioning in CDF.
  • ...and 1 more figures