Fiber Nonlinearity Mitigation in Coherent Optical Systems
Stella Civelli, Dario Cellini, Enrico Forestieri, Marco Secondini
TL;DR
This work addresses the impact of fiber Kerr nonlinearity on capacity in coherent optical systems by evaluating two complementary mitigation strategies: digital backpropagation (DBP) and constellation shaping. It surveys DBP, including low-complexity variants such as ESSFM and CB-ESSFM, and explores constellation shaping techniques like probabilistic amplitude shaping (PAS), sphere shaping, and sequence-based nonlinear shaping with sequence selection. The findings show that DBP can significantly mitigate intra-channel nonlinearity with reduced complexity, while PAS provides gains in the linear regime that degrade under carrier phase recovery (CPR); notably, sequence selection yields persistent nonlinear shaping gains even with CPR, albeit at higher computational cost. The results offer practical DSP guidelines and inform capacity-limit considerations for nonlinear fiber channels, highlighting trade-offs between performance and implementation feasibility.
Abstract
Fiber nonlinearity represents a critical challenge to the capacity enhancement of modern optical communication systems. In recent years, significant research efforts have focused on mitigating its impact through two complementary approaches. On the one hand, researchers have investigated practical digital signal processing (DSP) techniques to mitigate or compensate for nonlinear impairments, such as reversing fiber propagation effects through digital backpropagation (DBP). However, the high computational complexity of these techniques often discourages their practical implementation. On the other hand, information-theoretic studies have sought to establish the capacity limits of the nonlinear optical fiber channel, providing a framework for evaluating the ultimate performance of existing optical networks and guiding the design of next-generation systems. This work reviews recent advances and proposes future directions for nonlinearity compensation and mitigation, including constellation shaping techniques and low-complexity DBP. Furthermore, it highlights the potential of these innovations both in advancing the theoretical understanding of fiber capacity limits and in enabling practical DSP implementations.