During the COVID-19 pandemic, Twitter became a key platform for expressing public sentiment. This study explores a simple and scalable way to analyze the sentiment of COVID-19 tweets without the heavy computational cost of full transformer fine-tuning. Pretrained transformer models—Bidirectional Encoder Representations from Transformers (BERT), Distilled Bidirectional Encoder Representations from Transformers (DistilBERT), and Robustly Optimized BERT Pretraining Approach (RoBERTa)—were used in frozen mode to generate sentence embeddings, which were then classified using lightweight models: Support Vector Machine (SVM) and Logistic Regression (LR). On a balanced set of 7,500 labeled tweets from April–June 2020, the best hybrid model DistilBERT embeddings with Logistic Regression reached 0.64 accuracy and 0.62 F1-score (harmonic mean of precision and recall). For comparison, fully fine-tuned transformers on the larger 143,902-tweet dataset achieved up to 0.973 accuracy, zero-shot classifiers scored only 0.21–0.33, VADER reached 0.73, and TextBlob 0.45. The results show that combining frozen transformer embeddings with classical classifiers offers a practical middle ground: it delivers reasonable performance using only a central processing unit (CPU) and minimal training time, making it suitable for research groups or public-health teams with limited computational resources.
Rezaei et al. (Mon,) studied this question.