Next-slot OFDM-CSI Prediction: Multi-head Self-attention or State Space Model?

TL;DR

The paper tackles next-slot OFDM-CSI prediction in 5G NR by directly comparing two prominent DL layers: multi-head self-attention (MSA) and state-space models (SSM). Using UMi/UMa 3GPP channel models, it evaluates ID and OOD performance for both SISO and MIMO settings, revealing that SSMs generalize better in SISO but MSAs outperform SSMs in MIMO, with significant implications for computational efficiency and hardware deployment. The study emphasizes the trade-offs between predictive accuracy and resource usage, highlights the impact of mobility and SNR diversification on generalization, and provides open-source code to foster reproducibility and industry benchmarking. Overall, the results suggest selecting MSAs for MIMO-driven CSI prediction while recognizing SSMs’ potential advantages for simpler channels and hardware-constrained deployments, pointing toward hybrid or hybridized architectures for future AI-enabled 5G use cases.

Abstract

The ongoing fifth-generation (5G) standardization is exploring the use of deep learning (DL) methods to enhance the new radio (NR) interface. Both in academia and industry, researchers are investigating the performance and complexity of multiple DL architecture candidates for specific one-sided and two-sided use cases such as channel state estimation (CSI) feedback, CSI prediction, beam management, and positioning. In this paper, we set focus on the CSI prediction task and study the performance and generalization of the two main DL layers that are being extensively benchmarked within the DL community, namely, multi-head self-attention (MSA) and state-space model (SSM). We train and evaluate MSA and SSM layers to predict the next slot for uplink and downlink communication scenarios over urban microcell (UMi) and urban macrocell (UMa) OFDM 5G channel models. Our numerical results demonstrate that SSMs exhibit better prediction and generalization capabilities than MSAs only for SISO cases. For MIMO scenarios, however, the MSA layer outperforms the SSM one. While both layers represent potential DL architectures for future DL-enabled 5G use cases, the overall investigation of this paper favors MSAs over SSMs.

Figures (10)

• Figure 1: The input-output similarity of a neural network between (a) CSI prediction and (b) next token prediction.
• Figure 2: Illustration of the LTE radio frame structure
• Figure 3: SISO MSE of next-slot OFDM-CSI prediction task vs. test SNRs at $f_c=5$ GHz for multiple MSA layers in (a) and (c) and the SSM layers in (b) and (d) when each is trained with the UMi channel at different SNR values without a distribution shift in the UE speed (i.e., $v_{\textrm{train}} = v_{\textrm{test}}$).
• Figure 4: SISO MSE of next-slot OFDM-CSI prediction task vs. test SNRs at $f_c=5$ GHz for multiple MSA layers in (a) and (c) and the SSM layers in (b) and (d) when each is trained with the UMi channel at different SNR values with a distribution shift in the UE speed (i.e., $v_{\textrm{train}} \neq v_{\textrm{test}}$).
• Figure 5: MIMO MSE of next-slot OFDM-CSI prediction task vs. test SNRs at $f_c=5$ GHz for multiple MSA layers in (a) and (c) and the SSM layers in (b) and (d) when each is trained with the UMi channel at different SNR values without distribution shift in the UE speed (i.e., $v_{\textrm{train}} = v_{\textrm{test}}$).