Advanced signal detection in optical OFDM-VLC with 256-QAM scheme: LSTM-aided BER and PSD optimization over Rayleigh channels

This paper presents a comprehensive analysis of bit error rate (BER) and power spectrum density (PSD) performance for optical OFDM systems integrated with Visible Light Communication (VLC), employing various signal detection algorithms over Rayleigh fading channels. Simulation studies were conducted for 64, 256, and 512 subcarriers to evaluate BER under different SNR conditions. The proposed LSTM-based detection method achieved remarkable improvements, requiring only 6.1 dB, 8.1 dB, and 10.1 dB SNR, respectively, to attain a BER of 10 −5 , outperforming hybrid, QRM-MLD, MMSE, ZFE, and conventional optical OFDM by SNR gains ranging from 1.9 dB to 10.8 dB. Results also indicate that while increasing the number of subcarriers raises SNR demands, the LSTM detector consistently delivers superior performance, making it highly suitable for high-capacity advanced radio and VLC systems. Furthermore, PSD analysis demonstrates significant side lobe suppression, reducing out-of-band emissions from −100 dB down to −390 dB. This effective spectrum containment is crucial for minimizing spectral leakage in 5G, beyond-5G, and VLC frameworks, thereby enhancing spectrum utilization and overall system capacity. The study underscores the pivotal role of advanced detection methods like LSTM in improving throughput, reducing SNR requirements, and optimizing spectral efficiency for future optical OFDM-VLC communication systems.