July 15, 2025

Evaluation of Indobert and Roberta: Performance of Indonesian Language Transformer Models in Sentiment Classification

Key Points

IndoBERT achieved an accuracy of 70%, outperforming RoBERTa's 67%, indicating its effectiveness in sentiment classification.
The average F1-score of IndoBERT is 0.69, compared to RoBERTa's 0.65, suggesting better balance in classifying sentiment categories.
IndoBERT demonstrated lower evaluation loss values, indicating superior generalization capability in understanding the Indonesian context.
Faster and more stable training times were observed for IndoBERT, emphasizing its efficiency over RoBERTa.

Abstract

The development of Natural Language Processing (NLP) technology has had a significant impact on various fields, especially in sentiment analysis. This analysis becomes important in understanding public perception, especially on social media which has a lot of opinions. Indonesian, with its morphological complexity, dialectal variations, and dynamic everyday vocabulary usage, presents unique challenges in the development of NLP models. This study aims to evaluate and compare the performance of two Indonesian language transformer models, namely IndoBERT (Indonesia Bidirectional Encoder Representations from Transformers) and RoBERTa Indonesia (Robustly Optimized BERT Pretraining Approach) in applying sentiment classification using the Indonesian General Sentiment Analysis Dataset. Both models were fine-tuned using consistent hyperparameter configurations to ensure the validity of the comparison. Evaluation was conducted based on classification metrics, namely precision, recall, F1-score, and accuracy. The results show that the IndoBERT model excels in all aspects of evaluation. IndoBERT achieved an accuracy of 70%, while RoBERTa Indonesia only reached 67%. Additionally, the average F1-score of IndoBERT at 0.69 is higher compared to RoBERTa, which only reached 0.65. The performance of IndoBERT is also more balanced in classifying the three sentiment categories (negative, neutral, and positive), whereas RoBERTa shows less consistent performance, especially in negative and positive sentiments. In the loss analysis, IndoBERT produced a lower evaluation loss value, indicating better generalization capability. Additionally, IndoBERT also shows faster and more stable training times compared to RoBERTa. This performance difference shows that the architecture and pre-trained data used by each model affect their ability to understand Indonesian contextually. This research provides a comprehensive comparative overview of the effectiveness of two transformer models in the task of Indonesian language sentiment analysis, as well as lays the groundwork for selecting a more optimal model in the development of NLP systems for social media.

Mark Helpful

Bookmark

Relay