Constellation Design for Robust Interference Mitigation

TL;DR

The paper addresses symbol detection for single-carrier systems exposed to structured interference with Nakagami-$m$ statistics, showing that standard Euclidean detectors are suboptimal under joint random amplitude and nonuniform phase. It introduces the maximum-likelihood Gaussian-phase approximate (ML-G) detector, deriving a tractable metric via Gaussian phase moment matching and a convergent harmonic-series term $\mathcal{S}(r,\phi)$, and analyzes the resulting decision regions. Building on ML-G, it formulates and solves an SEP-minimizing constellation design under an average-energy constraint, using a global search with local refinement to produce interference-adaptive constellations that transition from near-hexagonal to asymmetric layouts as INR grows. Numerical results demonstrate consistent SEP gains over benchmark detectors in interference-dominated regimes, with Rayleigh interference ($m=1$) reducing to conventional Euclidean detection, validating both the detector and the constellation-design framework for practical spectrum-sharing contexts.

Abstract

This paper investigates symbol detection for single-carrier communication systems operating in the presence of additive interference with Nakagami-m statistics. Such interference departs from the assumptions underlying conventional detection methods based on Gaussian noise models and leads to detection mismatch that fundamentally affects symbol-level performance. In particular, the presence of random interference amplitude and non-uniform phase alters the structure of the optimal decision regions and renders standard Euclidean distance-based detectors suboptimal. To address this challenge, we develop the maximum-likelihood Gaussian-phase approximate (ML-G) detector, a low-complexity detection rule that accurately approximates maximum-likelihood detection while remaining suitable for practical implementation. The proposed detector explicitly incorporates the statistical properties of the interference and induces decision regions that differ significantly from those arising under conventional metrics. Building on the ML-G framework, we further investigate constellation design under interference-aware detection and formulate an optimization problem that seeks symbol placements that minimize the symbol error probability subject to an average energy constraint. The resulting constellations are obtained numerically and adapt to the interference environment, exhibiting non-standard and asymmetric structures as interference strength increases. Simulation results demonstrate clear symbol error probability gains over established benchmark schemes across a range of interference conditions, particularly in scenarios with dominant additive interference.

Figures (5)

• Figure 1: Decision regions for 8-PSK at $\gamma_S=10$ dB across various $\gamma_I$.
• Figure 2: Optimized constellations (minimum-SEP, ML-G metric) at fixed $\gamma_S=20$ dB, and $\gamma_I \in \{5,12,20\}$ dB, $m=2$.
• Figure 3: SEP versus $\gamma$ for $64$-QAM and $64$-PSK under various interference conditions.
• Figure 4: Maximum SEP difference between ML-G and CAI versus SNR across various QAM constellation orders.
• Figure 5: SEP versus $\gamma$ for various constellation designs.

Theorems & Definitions (6)

• Lemma 1
• Proposition 1
• Remark 1
• Proposition 2
• Proposition 3
• Proposition 4