Communication receiver design with pre-trained models based natural redundancy decoding

The current wireless communication system has higher demands for receivers’ performance. In this paper, Natural Redundancy (NR) which widely existed in the transmission sources is exploited to enhance the capability of current communication receivers in recovering information bit sequences from noisy measurements. To reveal the essential connotation of exploiting the NR in the sources to improve the performance of the communication receivers, we develop an NR coding and decoding theory by regarding the NR in the sources as a special kind of error-correcting code. Under this theory, we propose an effective “NR decoder + channel decoder” receiver structure. In addition, to sufficiently exploit the NR in the sources, we introduce the Pre-Trained Models (PTMs) into NR decoder design, which have a powerful capability to capture knowledge from massive data. Especially for uncompressed Chinese text sources, we design an efficient NR decoder based on Bidirectional Encoder Representation From Transformers (BERT) deep learning model in the Natural Language Processing (NLP) field. Comprehensive computer simulations were carried out and the results show that our proposed PTMs-based NR decoder has a significant effect on the performance improvement of the receiver and our designed receiver with the NR decoder can obtain over 2 dB performance gain compared with classical receiver on simplified Chinese edition Wiki-40B dataset.