Pragmatic intention detection in Amharic using transformer-based model

Understanding what a speaker intends to communicate is an important challenge in natural language processing, particularly when the intended meaning goes beyond the literal words of an utterance. Despite recent progress in multilingual language technologies, Amharic remains relatively underrepresented in research on pragmatic intention detection. This study investigates pragmatic intention detection in Amharic by developing a balanced, nine-class dataset and evaluating several machine learning and neural language models. The dataset contains 11,475 Amharic sentences covering Assertion, Complaint, Phatic, Promise, Refusal, Request, Sarcasm/Irony, Suggestion, and Warning, with 7,955 sentences used for training, 1,857 for validation, and 1,663 for testing. Three independent annotators labeled the dataset, achieving 99.90% agreement and a Fleiss' kappa of 0.9993, indicating very high annotation reliability. We compare TF-IDF with Support Vector Machine, BiLSTM, mBERT, XLM-R, and AfroXLM-R. Among these models, AfroXLM-R achieved the strongest performance on the held-out test set, with 79.98% accuracy and a macro-F1 score of 76.93%. Its performance was also significantly better than that of XLM-R according to McNemar’s exact test ( p < 0.001). These findings demonstrate the potential of multilingual transformer models for identifying pragmatic intentions in Amharic and provide a reproducible benchmark for further research. The study contributes an annotated resource and empirical baseline for advancing Amharic pragmatic natural language processing and developing more inclusive language technologies.