Low complexity neural network equalizer for nonlinear mitigation in digital subcarrier multiplexing systems

This paper presents a comparative study of neural network (NN) models for nonlinear compensation in digital subcarrier multiplexing (DSCM) systems. We employ the NN model based on bi-directional long short-term memory (biLSTM) and 1D-convolutional NN (1D-CNN) layers. To provide a lower complexity solution, weight clustering is applied to the NN. We specifically compare the performance of our proposed NN-based equalizer with traditional methods such as Chromatic Dispersion Compensation (CDC), Digital Back-Propagation (DBP) and alternative NN, based on triplet coefficients in perturbation analysis, proposed by Bakhshali, et al., evaluating both Q-factor performance and computational complexity. Our findings demonstrate that the NN-based equalizer offers competitive Q-factor improvements while significantly reducing the computational complexity, particularly when employing weight clustering. We show that the complexity of the NN can be reduced by up to 91.8% when compared to the NN based on perturbation analysis proposed by Bakhshali, et al. and by 31% when compared to the DBP with 1 step per span. The results underscore the potential of NN-based approaches to deliver high-performance nonlinear compensation with lower computational demands, making them a promising solution for future optical nonlinear equalizers.