When Do Transformers Actually Win? Sentiment Classification In Low-Data Settings

Nowadays, use of Transformer based language is increasing rapidly to perform NLP (Natural Language Processing) related tasks.DistilBERT is a model which provide strong classification performance can understand the context of sentences or words.The performance of Transformer usually depends on having labelled training data.But, in real world, obtaining labelled data is time consuming, expensive and difficult.This research focuses on "Does a Transformer always provide a meaningful advantage when only a limited amount of labelled data is present?"We are solving this question through a controlled sentiment-classification experiment which uses the Yelp Labelled Reviews dataset and contains customer reviews labelled as positive or negative in binary representation (0 & 1).Multinomial Naive Bayes, Logistic Regression and Linear SVM (Support Vector Machine) are the three traditional approaches used in this model.These models use TF-IDF (Term Frequency Inverse Document Frequency) to convert review text into numerical data called features.Further these results will be compared with DistilBERT which is a lightweight Transformer-based language model.A key factor of this research is using labelled training data in different proportions.The experiment will use five training conditions at 10%, 25%, 50%, 75% and 100%.This shows how the model will behave when the training data is Less or gradually increased.The performance of each model will be measured using precision, recall, accuracy, F-1 score.These measurements will not only evaluate model performance but also the computational cost behind it.The expected outcome of this research is practical understanding of the relationship between training data availability, computational cost and performance.The findings help to determine when a TF-IDF based model will be enough or using a Transformer model may help in improvement.This model will be proved useful when labelled data is limited.Overall, the study aims in making a choice between Traditional machine learning model or Transformer based model for sentiment classification.