Joint Rx IQ Imbalance Compensation and Timing Recovery for Faster-than-Nyquist WDM Systems

Faster-than-Nyquist (FTN) tight filtering introduces serious inter-symbol interference (ISI) impairment, leading to an insufficient compensation range for conventional IQ imbalance compensation algorithms. Furthermore, receiver (Rx) IQ imbalance and ISI impairments significantly increase the convergence cost required by the squared Gardner phase detector (SGPD) timing recovery algorithm to establish a timing synchronization loop. This paper proposes a joint Rx IQ compensation and timing recovery scheme. By embedding a two-stage IQ imbalance compensation algorithm into the timing recovery feedback loop, the proposed scheme could effectively estimate and compensate for Rx IQ imbalance. Meanwhile, thanks to the innovative scheme, which equalizes Rx IQ imbalance and ISI during the timing feedback loop, the convergence cost of timing recovery could be reduced compared with the conventional blind frequency domain (BFD) scheme. The simulation results of 128 GBaud polarization multiplexing (PM) 16-quadrature amplitude modulation (QAM) FTN wavelength division multiplexing (WDM) transmission systems demonstrate that the proposed scheme could bring about 14%, 12.5%, and 16.6% improvements in the compensation range for Rx IQ amplitude imbalance, phase imbalance, and skew, respectively, compared with the conventional one. Meanwhile, the convergence cost is reduced by at least 31% with a 0.9 acceleration factor. In addition, 40 GBaud PM-16QAM FTN experiment results show that the proposed scheme could bring about a 0.8 dB improvement in the optical signal noise ratio (OSNR) compared with the conventional BFD scheme.