Research on fine-tuning strategy of sentiment analysis model based on BERT

BERT, as a pre-trained model, can not only greatly improve the performance of the task models in the field of language processing, but also greatly save computational resources and costs. At present, most of sentiment classification tasks focus on the model structures. But it is very important to explore the changes of hyper-parameters based on the task models so as to obtain a general parameters setting method for improving the accuracy of models' predictions. In this paper, we conduct two parameter fine-tuning methods: static parameter fine-tune is used to improve the performance of the task models, and then use the layer frozen strategy to further fine-tune the task models. We propose that when the static learning rate is 2e-5 and the batch size is 32, the prediction performance of the sentiment analysis model is improved. Then, under such parameter settings, we further fine-tune the sentiment analysis model by adopting the dynamic layer frozen strategy. After fine-tuning, the prediction accuracy is more accurate than that under the static optimal parameters.