Benchmarking IndoBERT and Transformer Models for Sentiment Classification on Indonesian E-Government Service Reviews 1751 2427
DOI:
https://doi.org/10.26623/transformatika.v23i1.12095Keywords:
Analisis Sentimen, Pemrosesan Bahasa Alami, Indobert, Model Transformator, Penambangan DataAbstract
The rapid adoption of e-government services in Indonesia has increased the importance of understanding public sentiment toward digital platforms. This study presents a comparative analysis of five models—IndoBERT, mBERT, XLM-R, CNN, and BiLSTM—for sentiment classification on user reviews of NEWSAKPOLE, a public service application for vehicle tax and licensing. A custom dataset of 11,000+ reviews was scraped from the Google Play Store and labeled using a hybrid rating-based and manual validation approach. Each model was evaluated using accuracy, precision, recall, and F1-score. IndoBERT achieved the highest performance with an F1-score of 0.882, outperforming multilingual and classical deep learning models. Confusion matrix analysis showed that transformer-based models were more effective in detecting neutral and mixed sentiments, while CNN and BiLSTM struggled with misclassification. The results highlight IndoBERT's robustness in low-resource sentiment analysis and its potential to enhance public service monitoring and policy feedback mechanisms in Indonesian digital governance.
Downloads
References
[1] D. Marutho, Muljono, S. Rustad, and Purwanto, Optimizing aspect-based sentiment analysis using sentence embedding transformer, bayesian search clustering, and sparse attention mechanism, J. Open Innov. Technol. Mark. Complex., vol. 10, no. 1, p. 100211, 2024, doi: 10.1016/j.joitmc.2024.100211.
[2] C. Shaw, P. LaCasse, and L. Champagne, Exploring emotion classification of indonesian tweets using large scale transfer learning via IndoBERT, Soc. Netw. Anal. Min., vol. 15, no. 1, Mar. 2025, doi: 10.1007/s13278-025-01439-6.
[3] H. Imaduddin, F. Y. A’la, and Y. S. Nugroho, Sentiment Analysis in Indonesian Healthcare Applications using IndoBERT Approach, Int. J. Adv. Comput. Sci. Appl., vol. 14, no. 8, 2023, doi: 10.14569/ijacsa.2023.0140813.
[4] B. V. Kartika, M. J. Alfredo, and G. P. Kusuma, Fine-Tuned IndoBERT Based Model and Data Augmentation for Indonesian Language Paraphrase Identification, Rev. Intell. Artif., vol. 37, no. 3, pp. 733–743, Jun. 2023, doi: 10.18280/ria.370322.
[5] N. K. Nissa and E. Yulianti, Multi-label text classification of Indonesian customer reviews using bidirectional encoder representations from transformers language model, Int. J. Electr. Comput. Eng. IJECE, vol. 13, no. 5, p. 5641, Oct. 2023, doi: 10.11591/ijece.v13i5.pp5641-5652.
[6] S. Aras, M. Yusuf, R. Y. Ruimassa, E. A. B. Wambrauw, and E. B. Pala’langan, Sentiment Analysis on Shopee Product Reviews Using IndoBERT, J. Inf. Syst. Inform., vol. 6, no. 3, pp. 1616–1627, Sep. 2024, doi: 10.51519/journalisi.v6i3.814.
[7] F. Iscus and A. S. Girsang, Sentiment Analysis of COVID-19 Public Activity Restriction (PPKM) Impact using BERT Method, Int. J. Eng. Trends Technol., vol. 70, no. 12, pp. 281–288, Dec. 2022, doi: 10.14445/22315381/ijett-v70i12p226.
[8] Ibadurrohman Irfan Fatani, Twitter, Instagram, Youtube Speak: Understanding Sentiments on LRT Jabodebek Services via Inset Lexicon, IndoBERT and BERTopic Approaches, J. Electr. Syst., vol. 20, no. 4s, pp. 1028–1035, Apr. 2024, doi: 10.52783/jes.2147.
[9] S. Pecar, M. Simko, and M. Bielikova, Sentiment Analysis of Customer Reviews : Impact of Text Pre-processing, 2018 World Symp. Digit. Intell. Syst. Mach. DISA, pp. 251–256, 2018, doi: 10.1109/DISA.2018.8490619.
[10] R. K. Bania, COVID-19 Public Tweets Sentiment Analysis using TF-IDF and Inductive Learning Models Handwritten Assamese Character Recognition using Texture and Diagonal Orientation features with Artificial Neural Network View project COVID-19 Public Tweets Sentiment An, no. December, 2020, [Online]. Available: https://www.researchgate.net/publication/346572350
[11] D. R. Firmansyah and E. Lestariningsih, Analisis Sentimen Ulasan Aplikasi Smart Campus Unisbank di Google Playstore Menggunakan Algoritma Naive Bayes, J. JTIK J. Teknol. Inf. Dan Komun., vol. 8, no. 2, pp. 498–507, Apr. 2024, doi: 10.35870/jtik.v8i2.1882.
[12] D. Marutho, M. Muljono, R. Supriadi, and P. Purwanto, Optimizing Aspect Term Extraction and Sentiment Classification through Attention Mechanism and Sparse Attention Techniques, Int. J. Intell. Eng. Syst., vol. 17, no. 5, pp. 1004–1015, Oct. 2024, doi: 10.22266/ijies2024.1031.75.
[13] Primanda Sayarizki, Hasmawati, and H. Nurrahmi, Implementation of IndoBERT for Sentiment Analysis of Indonesian Presidential Candidates, Indones. J. Comput. Indo-JC, vol. Vol. 9 No. 2, pp. 61-72 Pages, Aug. 2024, doi: 10.34818/INDOJC.2024.9.2.934.
[14] E. Arif, S. Suherman, and A. P. Widodo, Predicting Stock Prices of Digital Banks: A Machine Learning Approach Combining Historical Data and Social Media Sentiment from X, Ingénierie Systèmes Inf., vol. 30, no. 3, Mar. 2025, doi: 10.18280/isi.300313.
[15] M. Rosidin, M. F. Gustafi, and S. A. Pratiwi, Optimizing nazief adriani’s stemmer algorithm in detecting indonesian word errors using sastrawi, no. 3.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Dhendra, Victor Gayuh Utomo
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
Transformatika is licensed under a Creative Commons Attribution 4.0 International License.
