Chinese named entity recognition with fusion of attention and whole word masking

In order to effectively obtain word-level semantic knowledge and address the issue of inaccurate extraction of salient information during feature extraction, this paper proposes a named entity recognition method that combines the RoBERTa-WWM (A Robustly Optimized BERT Pre-training Approach-Whole Word Masking) pretrained model with attention mechanisms. Firstly, the RoBERTa-WWM model is trained to obtain word-level semantic knowledge representation. This semantic representation is then sequentially inputted into a bidirectional long short-term memory (BiLSTM) network, where attention mechanisms are applied to assign weights to key information. Finally, a conditional random field (CRF) is used to obtain the globally optimal labels. Experimental results demonstrate that compared to traditional named entity algorithms that use character-level semantic knowledge, the proposed method achieves a 1.25% improvement in the F1 score. Moreover, further improvement of 2.06% in F1 score is achieved by weighting the key information, indicating the strong performance of the proposed model in Chinese named entity recognition task.