Superimposed-Pilot OTFS Under Fractional Doppler: Modular End-to-End Learning

TL;DR

This paper addresses OTFS performance under fractional Doppler by proposing a modular end-to-end learning framework that preserves OTFS structure while enabling joint optimization of transmission and reception. It replaces key blocks with trainable neural modules aligned to OTFS operations, including trainable constellation mapping, superimposed pilots, and learnable Zak/ Izak transforms, coupled with a JCED unit that explicitly accounts for CP and fractional Doppler. The JCED module combines a Pilot Separation Network, a CENet-based channel estimator, and a differentiable LMMSE detector to achieve end-to-end BER performance close to the perfect CSI bound, while reducing pilot overhead compared to EP-based schemes. Simulation results demonstrate improved NMSE and detection reliability across integer and fractional Doppler, with robust generalization to larger OTFS grids, highlighting the practicality of modular DL-driven transceivers for high-mobility networks.

Abstract

Orthogonal time frequency space (OTFS) modulation has emerged as a promising candidate to overcome the performance degradation of orthogonal frequency division multiplexing (OFDM), which are commonly encountered in high-mobility wireless communication scenarios. However, conventional OTFS transceivers rely on multiple separately designed signal-processing modules, whose isolated optimization often limits global optimal performance. To overcome limitations, this paper proposes a modular deep learning (DL) based end-to-end OTFS transceiver framework that consists of trainable and interchangeable neural network (NN) modules, including constellation mapping/demapping, superimposed pilot placement, inverse Zak (IZak)/Zak transforms, and a U-Net-enhanced NN tailored for joint channel estimation and detection (JCED), while explicitly accounting for the impact of the cyclic prefix. This physics-informed modular architecture provides flexibility for integration with conventional OTFS systems and adaptability to different communication configurations. Simulations demonstrate that the proposed design significantly outperforms baseline methods in terms of both normalized mean squared error (NMSE) and detection reliability, maintaining robustness under integer and fractional Doppler conditions. The results highlight the potential of DL-based end-to-end optimization to enable practical and high-performance OTFS transceivers for next-generation high-mobility networks.

Figures (12)

• Figure 1: OTFS system model.
• Figure 2: Equivalent channel matrices of integer and fractional Doppler.
• Figure 3: The model design of (a) DL-IZak and (b) DL-Zak.
• Figure 4: The network structure of the proposed JCED.
• Figure 5: The detailed structure of the CENet.