Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Beamforming with Doppler-Aware Sparse Attention for High Mobility Environments

Cemil Vahapoglu, Timothy J. O'Shea, Wan Liu, Sennur Ulukus

TL;DR

Simulation results under urban macro (UMa) channel conditions show that Doppler-aware Sparse NNBF significantly outperforms both a fixed-pattern baseline, referred to as Standard Sparse NNBF, and conventional beamforming techniques ZFBF and MMSE beamforming in high mobility scenarios, while maintaining structured sparsity with a controlled number of attended keys per query.

Abstract

Beamforming has significance for enhancing spectral efficiency and mitigating interference in multi-antenna wireless systems, facilitating spatial multiplexing and diversity in dense and high mobility scenarios. Traditional beamforming techniques such as zero-forcing beamforming (ZFBF) and minimum mean square error (MMSE) beamforming experience performance deterioration under adverse channel conditions. Deep learning-based beamforming offers an alternative with nonlinear mappings from channel state information (CSI) to beamforming weights by improving robustness against dynamic channel environments. Transformer-based models are particularly effective due to their ability to model long-range dependencies across time and frequency. However, their quadratic attention complexity limits scalability in large OFDM grids. Recent studies address this issue through sparse attention mechanisms that reduce complexity while maintaining expressiveness, yet often employ patterns that disregard channel dynamics, as they are not specifically designed for wireless communication scenarios. In this work, we propose a Doppler-aware Sparse Neural Network Beamforming (Doppler-aware Sparse NNBF) model that incorporates a channel-adaptive sparse attention mechanism in a multi-user single-input multiple-output (MU-SIMO) setting. The proposed sparsity structure is configurable along 2D time-frequency axes based on channel dynamics and is theoretically proven to ensure full connectivity within p hops, where p is the number of attention heads. Simulation results under urban macro (UMa) channel conditions show that Doppler-aware Sparse NNBF significantly outperforms both a fixed-pattern baseline, referred to as Standard Sparse NNBF, and conventional beamforming techniques ZFBF and MMSE beamforming in high mobility scenarios, while maintaining structured sparsity with a controlled number of attended keys per query.

Neural Beamforming with Doppler-Aware Sparse Attention for High Mobility Environments

TL;DR

Simulation results under urban macro (UMa) channel conditions show that Doppler-aware Sparse NNBF significantly outperforms both a fixed-pattern baseline, referred to as Standard Sparse NNBF, and conventional beamforming techniques ZFBF and MMSE beamforming in high mobility scenarios, while maintaining structured sparsity with a controlled number of attended keys per query.

Abstract

Beamforming has significance for enhancing spectral efficiency and mitigating interference in multi-antenna wireless systems, facilitating spatial multiplexing and diversity in dense and high mobility scenarios. Traditional beamforming techniques such as zero-forcing beamforming (ZFBF) and minimum mean square error (MMSE) beamforming experience performance deterioration under adverse channel conditions. Deep learning-based beamforming offers an alternative with nonlinear mappings from channel state information (CSI) to beamforming weights by improving robustness against dynamic channel environments. Transformer-based models are particularly effective due to their ability to model long-range dependencies across time and frequency. However, their quadratic attention complexity limits scalability in large OFDM grids. Recent studies address this issue through sparse attention mechanisms that reduce complexity while maintaining expressiveness, yet often employ patterns that disregard channel dynamics, as they are not specifically designed for wireless communication scenarios. In this work, we propose a Doppler-aware Sparse Neural Network Beamforming (Doppler-aware Sparse NNBF) model that incorporates a channel-adaptive sparse attention mechanism in a multi-user single-input multiple-output (MU-SIMO) setting. The proposed sparsity structure is configurable along 2D time-frequency axes based on channel dynamics and is theoretically proven to ensure full connectivity within p hops, where p is the number of attention heads. Simulation results under urban macro (UMa) channel conditions show that Doppler-aware Sparse NNBF significantly outperforms both a fixed-pattern baseline, referred to as Standard Sparse NNBF, and conventional beamforming techniques ZFBF and MMSE beamforming in high mobility scenarios, while maintaining structured sparsity with a controlled number of attended keys per query.

Paper Structure

This paper contains 18 sections, 2 theorems, 20 equations, 7 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. Uplink Multi-User SIMO (MU-SIMO) Setup
  4. Beamforming Design for Sum-Rate Maximization Problem
  5. Deep Neural Network (DNN) Architecture
  6. Overall Model Structure
  7. Separable Grouped Convolutional Network
  8. Stacked Multi-Channel Attention
  9. Training Procedure
  10. Doppler-Aware Sparse Attention Mechanism
  11. Proposed Sparsification Structure
  12. Multi-Head Attention Graph
  13. Intra-class connectivity
  14. Inter-class bridging
  15. Inter-Class Connectivity via Linear Congruence
  16. ...and 3 more sections

Key Result

Lemma 1

Let $s = \lceil T^{1-1/p} \rceil$ denote the stride of the global head ($h = 0$). Then, the attention graph $\mathcal{G}_0$ corresponding to Head 0 partitions the node set $\mathcal{V} = \{0, 1, \dots, T{-}1\}$ into $s$ disjoint equivalence classes: Each equivalence class $C_r$ generates a complete subgraph in $\mathcal{G}_0$. There are no edges between nodes of distinct classes, i.e., $\mathcal{

Figures (7)

  • Figure 1: Uplink multi-user SIMO system in a dense urban environment, where single-antenna UEs transmit data streams on the same time/frequency resources and the $M$-antenna BS applies digital beamforming on the received signal $\mathbf{y}$.
  • Figure 2: Deep neural network architecture.
  • Figure 3: Doppler-aware sparsification structure for a given query index $(l_q,k_q)=(7,32)$ when the number of heads $p$ is $2$.
  • Figure 4: Fixed strided sparsification structure child2019 for a given query index $i= 7\cdot 48 + 32$ when number of heads $p$ is 2 and fixed stride $s$ is $\lceil T^{1-\frac{1}{p}}\rceil$.
  • Figure 5: Performance comparison under low Doppler conditions $[v_{\min}, v_{\max}] = [0,10]\,\mathrm{m/s}$. Doppler-aware and standard sparse NNBF methods perform similarly and match baseline methods ZFBF and MMSE in both (a) average sum-rate and (b) BLER.
  • ...and 2 more figures

Theorems & Definitions (4)

  • Lemma 1: Partitioning by Global Head
  • proof
  • Theorem 1: Full Connectivity with Global Head
  • proof : Proof Sketch