Text classification as a sentiment analysis, spam and document classification is an important task in Natural Language Processing (NLP). Recent developments of deep learning, and transformer-based models have contributed significantly to classifying accuracy. These models have difficulty meeting computational efficiency, inference latency, and interpretability criteria, and cannot be used in resource-constrained and real-time settings. The models are critically compared based on accuracy, computational efficiency and trade-offs on benchmark datasets in this paper. The results indicate that transformer models are much more accurate than other conventional concepts, but have a significant disadvantage of being computationally expensive, which severely limits their actual implementation. Also, XAI methods and fairness-conscious training approaches are required particularly when issues of transparency and fairness of the existing model emerge. New methods like zero-shot learning, few-shot learning, and self-supervised learning are promising directions to further improve generalization and flexibility of NLP models. Future studies must focus on identifying light, energy efficient NLP architecture that can achieve good accuracy, interpretability and scalability. This work fills the gaps between theory and practice, which is why it can be of interest to both academic researchers and practitioners in the industry.
Jain et al. (Thu,) studied this question.