Signal detection in optical orthogonal time space modulation for efficient VLC application

This paper presents a detailed analysis of the bit error rate (BER) performance of a proposed maximum likelihood (ML) detection scheme for optical orthogonal time space modulation (OTSM) systems using 64-QPSK, considering practical optical impairments and diverse channel conditions. The evaluation covers scenarios with 5 % and 10 % channel estimation errors, as well as Rayleigh and Rician fading environments. Simulation results confirm that the proposed machine learning (ML) detector consistently outperforms conventional methods – including OTSM, zero-forcing equalization (ZFE), minimum mean square error (MMSE), and conventional ML – by delivering substantial SNR gains. For instance, under 10 % and 5 % estimation errors, the target BER of 10 −3 is achieved at 12.2 dB and 10.8 dB, respectively, providing up to 6 dB improvement over baselines. In Rayleigh fading, the same BER is attained at 9.6 dB with a gain of 7.7 dB, while in Rician fading, the detector achieves optimal performance at only 6 dB, outperforming others by as much as 9.5 dB. These results underscore the robustness of the proposed ML approach against estimation inaccuracies and fading, making it well-suited for low-power, high-reliability applications in 6G, Internet of things (IoT), vehicular networks, and satellite communications.