Analisis Sentimen Ulasan Aplikasi PLN Mobile Menggunakan Naïve Bayes, SVM, dan Random Forest dengan SMOTE 21 12
DOI:
https://doi.org/10.26623/transformatika.v24i1.14934Keywords:
Machine Learning, PLN Mobile, sentiment analysis, smote, TF-IDFAbstract
The PLN Mobile application, as the digital service platform of PT PLN (Persero), receives thousands of user reviews on the Google Play Store, making manual sentiment analysis impractical. This study aims to compare the performance of three machine learning algorithms Naïve Bayes, Support Vector Machine (SVM), and Random Forest in classifying PLN Mobile user reviews into two sentiment categories: positive and negative. These algorithms were selected because they represent three distinct classification paradigms: probabilistic, margin-based, and ensemble learning, all of which have demonstrated effectiveness in text classification tasks. The novelty of this study lies in the integration of a balanced data collection strategy based on rating categories, class balancing using the Synthetic Minority Over-sampling Technique (SMOTE), 5-fold cross-validation, and SVM hyperparameter tuning. A total of 3,702 reviews were collected through web scraping. After text preprocessing and TF-IDF feature weighting, 2,917 reviews (1,586 negative and 1,331 positive) with 4,160 features were retained for analysis. The cross-validation results showed that Naïve Bayes achieved the highest average F1-score of 0.846 with a low standard deviation of 0.011, indicating stable performance. Testing on 584 unseen reviews further confirmed that Naïve Bayes outperformed the other models, achieving 84% accuracy, precision, recall, and F1-score, followed by SVM (83%) and Random Forest (81%). Word frequency analysis revealed that users primarily appreciated the application's convenience and service efficiency, while the most common complaints concerned prolonged power outages and issues related to payment and complaint-handling processes. These findings demonstrate the superiority of Naïve Bayes for sentiment classification of relatively simple Indonesian-language user reviews and provide practical insights for improving the quality of PLN Mobile services.
Downloads
References
M. D. Rizkiyanto, M. D. Purbolaksono, and W. Astuti, “Sentiment Analysis Classification on PLN Mobile Application Reviews using Random Forest Method and TF-IDF Feature Extraction,” INTEK: Jurnal Penelitian, vol. 11, no. 1, pp. 37–43, Apr. 2024, doi: 10.31963/intek.v11i1.4774.
U. Brawijaya, K. R. Hilal, N. Y. Setiawan, and D. E. Ratnawati, “Fakultas Ilmu Komputer ANALISIS SENTIMEN BERBASIS ASPEK UNTUK PENGGUNA PLN MOBILE PADA GOOGLE PLAYSTORE MENGGUNAKAN METODE SUPPORT VECTOR MACHINE (SVM),” 2017. [Online]. Available: http://j-ptiik.ub.ac.id
S. Syafrizal, M. Afdal, and R. Novita, “Analisis Sentimen Ulasan Aplikasi PLN Mobile Menggunakan Algoritma Naïve Bayes Classifier dan K-Nearest Neighbor,” MALCOM: Indonesian Journal of Machine Learning and Computer Science, vol. 4, no. 1, pp. 10–19, Dec. 2023, doi: 10.57152/malcom.v4i1.983.
B. A. Prabowo, A. Hindasyah, and A. Khalid Rivai, “SENTIMENT ANALYSIS OF PLN MOBILE APPLICATION SERVICES USING NAIVE BAYES, SUPPORT VECTOR MACHINE (SVM) AND DECISION TREE METHODS,” Jurnal Riset Informatika, vol. 7, no. 3, pp. 236–243, Jun. 2025, doi: 10.34288/jri.v7i3.378.
J. M. Ayomi, A. V. Vitianingsih, Y. Kristyawan, A. L. Maukar, and T. Widiartin, “Sentiment Analysis of User Reviews for the PLN Mobile Application Using Naïve Bayes and Long Short-Term Memory,” Journal of Information Systems and Informatics, vol. 7, no. 4, pp. 3849–3873, Dec. 2025, doi: 10.63158/journalisi.v7i4.1342.
M. Arya Java, M. Syafrullah, and F. Teknologi, “Analisis Sentimen Ulasan Pengguna Aplikasi Threads pada Google Play Store Menggunakan Multinomial Naive Bayes dan Support Vector Machine,” Jurnal TICOM: Technology of Information and Communication, vol. 12, no. 2, 2024, [Online]. Available: https://github.com/nasalsabila/kamus-alay
N. D. Kurniawan, P. R. Ferdian, and N. Hidayati, “Analisis Sentimen Algoritma Naïve Bayes, Support Vector Machine, dan Random Forest Pada Ulasan Aplikasi Ajaib,” Jurnal Nasional Teknologi dan Sistem Informasi, vol. 11, no. 1, pp. 87–97, May 2025, doi: 10.25077/teknosi.v11i1.2025.87-97.
S. A. S. Mola, D. L. B. Baun, I. O. Nunes, and M. M. A. R. Sani, “ANALISIS SENTIMEN APLIKASI HALO BCA DI GOOGLE PLAY STORE MENGGUNAKAN METODE NAIVE BAYES, SUPPORT VECTOR MACHINE DAN RANDOM FOREST,” HOAQ (High Education of Organization Archive Quality) : Jurnal Teknologi Informasi, vol. 15, no. 2, pp. 69–79, Dec. 2024, doi: 10.52972/hoaq.vol15no2.p69-79.
M. Winda, P. 1 Ai, I. Warnilah, B. K. Simpony, V. Christian, and M. Dhafa Alfareza, “Implementasi Algoritma Model Random Forest, SVM Dan Naive Bayes Untuk Sentimen Analisis Aplikasi Gojek Di Playstore,” Jurnal Sains dan Manajemen, vol. 13, no. 2, 2025.
N. S. Sediatmoko, Y. Nataliani, and I. Suryady, “Suryady (Sentiment Analysis of Customer Review Using Classification Algorithms and SMOTE for Handling Imbalanced Class),” 2024.
I. G. B. A. Budaya and I. K. P. Suniantara, “Comparison of Sentiment Analysis Algorithms with SMOTE Oversampling and TF-IDF Implementation on Google Reviews for Public Health Centers,” MALCOM: Indonesian Journal of Machine Learning and Computer Science, vol. 4, no. 3, pp. 1077–1086, Jul. 2024, doi: 10.57152/malcom.v4i3.1459.
D. Andriyani, A. Faqih, and S. E. Permana, “Journal of Artificial Intelligence and Engineering Applications The Effect of SMOTE Application on Support Vector Machine Performance in Sentiment Classification on Imbalanced Datasets,” 2025. [Online]. Available: https://ioinformatic.org/
E. A. Lisangan et al., “Implementasi Naive Bayes pada Analisis Sentimen Opini Masyarakat di Twitter Terhadap Kondisi New Normal di Indonesia,” 2022.
T. Kemendikbud et al., “J-INTECH (Journal of Information and Technology) Analisis Sentimen Review Pelanggan Lazada dengan Sastrawi Stemmer dan SVM-PSO untuk Memahami Respon Pengguna”.
A. Gholamy, V. Kreinovich, O. Kosheleva, A. Gholamy, V. Kreinovich, and O. Kosheleva, “Why 70/30 or 80/20 Relation Between Training and Testing Sets: A Pedagogical Explanation,” Departmental Technical Reports (CS), Feb. 2018, Accessed: Jul. 20, 2026. [Online]. Available: https://scholarworks.utep.edu/cs_techrep/1209
R. Marta Dinata, E. Rayhana, V. Hadi, and U. Alkaf, “ANALISIS KOMPREHENSIF KINERJA MODEL KLASIFIKASI SENTIMEN: EVALUASI LINTAS METRIK PADA DATASET TWEET FILM BAHASA INDONESIA,” Jurnal Rekayasa Informasi, vol. 14, no. 1, p. 2025.
N. Hidayah and D. Dodiman, “Implementasi Algoritma Multinomial Naïve Bayes, TF-IDF dan Confusion Matrix dalam Pengklasifikasian Saran Monitoring dan Evaluasi Mahasiswa Terhadap Dosen Teknik Informatika Universitas Dayanu Ikhsanuddin,” Jurnal Akademik Pendidikan Matematika, pp. 8–15, May 2024, doi: 10.55340/japm.v10i1.1491.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Helmy Agta Al Fatah, Rara Sriartati Redjeki
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
Transformatika is licensed under a Creative Commons Attribution 4.0 International License.
