Sentiment Classification in Low-Resource, Code-Mixed Languages: Comparative Study of Llama 3 and Other Large Language Models

Large language models have seen limited evaluation on code-switched languages, despite the increasing use of such linguistic patterns on social media platforms. This study presents a systematic evaluation of Llama 3 for sentiment analysis of "Banglish", the informal blend of Bengali and English widely used online in Bangladesh. We examine five adaptation strategies: zero-shot prompting, few-shot learning, a two-step translation and analysis pipeline, fine-tuning, and model ensembling. Our experiments, conducted on 11, 673 posts from the BengaliBanglish₈0K dataset, show that fine-tuning Llama 3 delivers the best performance, achieving an accuracy of 66. 87 percent. This result outperforms its own zero-shot (43. 80 percent) and few-shot (49. 14 percent) baselines, as well as the zero-shot results of GPT-3. 5 (55. 90 percent), GPT-4 (65. 15 percent), and Claude 3. 5 (47. 68 percent). The dual-phase pipeline achieved 48. 07 percent accuracy, and the ensemble method reached 66. 78 percent, offering slight improvements but falling short of the fine-tuned model. We release our best-performing model, samiur-r/BanglishSentiment-Llama3-8B, to support further research. These findings highlight the effectiveness of task-specific fine-tuning in low-resource, code-mixed settings and emphasize the need for more comprehensive code-switching datasets and pre-training strategies tailored to linguistically diverse communities.