Roman Urdu Sentiment Analysis

This report studies four-class sentiment and abuse classification of Roman Urdu text (Negative, Positive, Neutral, Abusive). Roman Urdu is Urdu written in the Latin script, usually mixed with English and spelled phonetically with no fixed standard, which makes it difficult for models built for standardized languages. A public Roman Urdu sentiment dataset and the RUHSOLD hate-speech dataset were merged under a unified label scheme. The Abusive class was heavily underrepresented in training (3,358 samples against more than 20,000 per other class), so dictionary-based synonym replacement was used to expand it to 13,374 samples. Augmentation was applied to the training split only. Three models were compared: an LSTM trained from scratch, fine-tuned multilingual BERT (mBERT), and fine-tuned GPT-2. On the unaugmented test set, GPT-2 performed best with 70.24% accuracy and a macro F1 of 0.65, followed by mBERT (65.05%, 0.62) and the LSTM (57.43%, 0.54). Abusive remained the hardest class for every model (F1 0.31–0.42). The results show that pre-trained transformers transfer usefully to Roman Urdu, and that an English-only model can outperform a multilingual one on Latin-script, code-mixed text. This is a course project report for Natural Language Processing at Habib University, Karachi, Pakistan. It has not been peer-reviewed. Code is available at https://github.com/aqib420/RomanUrduSentimentAnalysis.