Deep Reinforcement Learning for MIMO Communication with Low-Resolution ADCs

TL;DR

The paper tackles the power-hungry ADC problem in mmWave MIMO by proposing a deep reinforcement learning framework that jointly optimizes the receiver front-end: the analog combiner ($\mathbf{v}$), ADC thresholds ($\mathbf{t}$), and discrete input alphabet ($\mathcal{X}$). It integrates differentiable mutual-information estimators (e.g., CORTICAL) to provide rewards and employs policy-gradient learning with a KL-penalty to train a stable policy, aiming to maximize the mutual information $I(\mathbf{X}; \widehat{\mathbf{W}})$ under a power constraint. The authors provide a theoretical convergence analysis showing that greedy policies over a truncated, discretized MDP converge to the optimal policy as the discretization becomes finer and the truncation bound grows, and they demonstrate through extensive simulations that the RL approach approaches the performance of exhaustive search while offering significant computational savings. The work demonstrates robustness to dynamic channel statistics and noisy CSI, highlighting practical significance for scalable, high-throughput MIMO systems with low-resolution quantization.

Abstract

Multiple-input multiple-output (MIMO) wireless systems conventionally use high-resolution analog-to-digital converters (ADCs) at the receiver side to faithfully digitize received signals prior to digital signal processing. However, the power consumption of ADCs increases significantly as the bandwidth is increased, particularly in millimeter wave communications systems. A combination of two mitigating approaches has been considered in the literature: i) to use hybrid beamforming to reduce the number of ADCs, and ii) to use low-resolution ADCs to reduce per ADC power consumption. Lowering the number and resolution of the ADCs naturally reduces the communication rate of the system, leading to a tradeoff between ADC power consumption and communication rate. Prior works have shown that optimizing over the hybrid beamforming matrix and ADC thresholds may reduce the aforementioned rate-loss significantly. A key challenge is the complexity of optimization over all choices of beamforming matrices and threshold vectors. This work proposes a reinforcement learning (RL) architecture to perform the optimization. The proposed approach integrates deep neural network-based mutual information estimators for reward calculation with policy gradient methods for reinforcement learning. The approach is robust to dynamic channel statistics and noisy CSI estimates. It is shown theoretically that greedy RL methods converge to the globally optimal policy. Extensive empirical evaluations are provided demonstrating that the performance of the RL-based approach closely matches exhaustive search optimization across the solution space.

Figures (5)

• Figure 1: Overview of the MIMO communication system.
• Figure 2: Overview of the proposed reinforcement learning model, where the environment is defined by $\mathbf{H}$ and the initial state consisting of analog processing matrix $\mathbf{v}$, thresholds $\mathbf{t}$, and input alphabet $\mathcal{X}$. The policy then takes the current state as input and outputs a distribution with mean $\mu$ and covariance $\Sigma$ from which an action is chosen to determine the next state. Finally, the reward is computed by a mutual information estimator.
• Figure 3: Capacity as a function of SNR in various communication scenarios.
• Figure 4: Input values (points) and threshold values (lines) for SISO with $n_q=3$. Point brightness indicates input probability.
• Figure 5: Learned constellations in MIMO with $n_r=2$ and $n_q=4$. Point brightness indicates input probability.

Theorems & Definitions (1)

• Theorem 1