Explainable Stacked Ensemble Learning for Fake News Detection using Text and Metadata 19 7
DOI:
https://doi.org/10.26623/themessenger.v18i1.16007Keywords:
Fake News Detection, Explainable Artificial Intelligence, Stacked Ensemble Learning, Text-Metadata Disagreement, Speaker CredibilityAbstract
Purpose: Automated fake news detection can support verification, but predictions are less useful when textual and contextual evidence cannot be inspected. The study examined whether combining claim text with speaker metadata could improve veracity classification while retaining explanations for journalistic review.
Methods: The LIAR train, validation, and test partitions were used, with six veracity labels remapped into Fake and Real classes. A BERT-base text classifier and Extreme Gradient Boosting metadata classifier produced class probabilities combined by an L1-regularized logistic regression meta-learner with disagreement and interaction features. SHapley Additive exPlanations, Local Interpretable Model-agnostic Explanations, and Integrated Gradients inspected metadata, ensemble, and token-level behavior.
Findings: The stacking model reached 0.7514 accuracy, 0.7483 macro-F1, 0.8288 area under the receiver operating characteristic curve, and 0.4970 Matthews correlation coefficient. It exceeded the BERT text branch (macro-F1=0.6203) and metadata branch (macro-F1=0.7066). Metadata provided the stronger signal, with credibility score and speaker-history variables receiving the largest SHAP importance. High text-metadata disagreement occurred in 30.4% of test cases, and the ensemble reached 0.8234 accuracy within this subset.
Originality: The study extends explainable fake-news detection by conceptualizing disagreement between evidence sources as a dimension of interpretability, separating semantic claim evidence, contextual speaker-history evidence, and their interaction at the fusion stage.
Downloads
References
Adadi, A., & Berrada, M. (2018). Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI). IEEE Access, 6, 52138–52160. https://doi.org/10.1109/ACCESS.2018.2870052
Alghamdi, J., Lin, Y., & Luo, S. (2024). Unveiling the hidden patterns: A novel semantic deep learning approach to fake news detection on social media. Engineering Applications of Artificial Intelligence, 137, 109240. https://doi.org/10.1016/j.engappai.2024.109240
Alhindi, T., Petridis, S., & Muresan, S. (2018). Where is Your Evidence: Improving Fact-checking by Justification Modeling. In J. Thorne, A. Vlachos, O. Cocarascu, C. Christodoulopoulos, & A. Mittal (Eds.), Proceedings of the First Workshop on Fact Extraction and VERification (FEVER) (pp. 85–90). Association for Computational Linguistics. https://doi.org/10.18653/v1/W18-5513
Aslam, Z., Missen, M. M. S., Ghaffar, A. A., Mehmood, A., Villar, M. G., Alvarado, E. S., & Ashraf, I. (2025). Advancing fake news combating using machine learning: A hybrid model approach. Knowledge and Information Systems, 67(12), 12137–12177. https://doi.org/10.1007/s10115-025-02588-y
Athira, A. B., Kumar, S. D. M., & Chacko, A. M. (2023). A systematic survey on explainable AI applied to fake news detection. Engineering Applications of Artificial Intelligence, 122, 106087. https://doi.org/10.1016/j.engappai.2023.106087
Bennett, W. L., & Livingston, S. (2018). The disinformation order: Disruptive communication and the decline of democratic institutions. European Journal of Communication, 33(2), 122–139. https://doi.org/10.1177/0267323118760317
Broussard, M., Diakopoulos, N., Guzman, A. L., Abebe, R., Dupagne, M., & Chuan, C.-H. (2019). Artificial Intelligence and Journalism. Journalism & Mass Communication Quarterly, 96(3), 673–695. https://doi.org/10.1177/1077699019859901
Calvo-Rubio, L.-M., & Ufarte-Ruiz, M.-J. (2021). Artificial intelligence and journalism: Systematic review of scientific production in Web of Science and Scopus (2008-2019). Communication & Society, 159–176. https://doi.org/10.15581/003.34.2.159-176
Cao, J., Zhuo, S., Su, J., & Chen, G. (2025). A Fake News Detection Model Based on Capsule Networks and Collaborative Attention. Applied Sciences, 15(22), 12190. https://doi.org/10.3390/app152212190
Chalehchaleh, R., Salehi, M., Farahbakhsh, R., & Crespi, N. (2024). BRaG: A hybrid multi-feature framework for fake news detection on social media. Social Network Analysis and Mining, 14(1), 35. https://doi.org/10.1007/s13278-023-01185-7
Chen, T., & Guestrin, C. (2016). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, 785–794. https://doi.org/10.1145/2939672.2939785
Collins, B., Hoang, D. T., Nguyen, N. T., & Hwang, D. (2021). Trends in combating fake news on social media – a survey. Journal of Information and Telecommunication, 5(2), 247–266. https://doi.org/10.1080/24751839.2020.1847379
De Magistris, G., Russo, S., Roma, P., Starczewski, J. T., & Napoli, C. (2022). An Explainable Fake News Detector Based on Named Entity Recognition and Stance Classification Applied to COVID-19. Information, 13(3), 137. https://doi.org/10.3390/info13030137
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In J. Burstein, C. Doran, & T. Solorio (Eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (pp. 4171–4186). Association for Computational Linguistics. https://doi.org/10.18653/v1/N19-1423
Farnoush, A., Gupta, A., Li, H., Benitez, J., & Jiang, W. (Kayla). (2026). Towards developing fake and satire news detection policies using component-based SEM and interpersonal detection theory. Information & Management, 63(2), 104293. https://doi.org/10.1016/j.im.2025.104293
Guess, A. M., Nyhan, B., & Reifler, J. (2020). Exposure to untrustworthy websites in the 2016 US election. Nature Human Behaviour, 4(5), 472–480. https://doi.org/10.1038/s41562-020-0833-x
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2018). A Survey of Methods for Explaining Black Box Models. ACM Computing Surveys (CSUR), 51(5), 93:1-93:42. https://doi.org/10.1145/3236009
Gutiérrez-Caneda, B., Lindén, C.-G., & Vázquez-Herrero, J. (2024). Ethics and journalistic challenges in the age of artificial intelligence: Talking with professionals and experts. Frontiers in Communication, 9. https://doi.org/10.3389/fcomm.2024.1465178
Hashmi, E., Yayilgan, S. Y., Yamin, M. M., Ali, S., & Abomhara, M. (2024). Advancing Fake News Detection: Hybrid Deep Learning With FastText and Explainable AI. IEEE Access, 12, 44462–44480. https://doi.org/10.1109/ACCESS.2024.3381038
Hu, B., Mao, Z., & Zhang, Y. (2025). An overview of fake news detection: From a new perspective. Fundamental Research, 5(1), 332–346. https://doi.org/10.1016/j.fmre.2024.01.017
Jadhav, R., Meshram, V., Bhosle, A., Patil, K., Dash, S., & Jadhav, S. (2025). Explainable multilingual and multimodal fake-news detection: Toward robust and trustworthy AI for combating misinformation. Frontiers in Artificial Intelligence, 8. https://doi.org/10.3389/frai.2025.1690616
Kaliyar, R. K., Goswami, A., & Narang, P. (2021). FakeBERT: Fake news detection in social media with a BERT-based deep learning approach. Multimedia Tools and Applications, 80(8), 11765–11788. https://doi.org/10.1007/s11042-020-10183-2
Koloski, B., Stepišnik Perdih, T., Robnik-Šikonja, M., Pollak, S., & Škrlj, B. (2022). Knowledge graph informed fake news classification via heterogeneous representation ensembles. Neurocomputing, 496, 208–226. https://doi.org/10.1016/j.neucom.2022.01.096
Kozik, R., Ficco, M., Pawlicka, A., Pawlicki, M., Palmieri, F., & Choraś, M. (2024). When explainability turns into a threat—Using xAI to fool a fake news detection method. Computers & Security, 137, 103599. https://doi.org/10.1016/j.cose.2023.103599
Kula, S., Kozik, R., & Choraś, M. (2022). Implementation of the BERT-derived architectures to tackle disinformation challenges. Neural Computing and Applications, 34(23), 20449–20461. https://doi.org/10.1007/s00521-021-06276-0
Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., Metzger, M. J., Nyhan, B., Pennycook, G., Rothschild, D., Schudson, M., Sloman, S. A., Sunstein, C. R., Thorson, E. A., Watts, D. J., & Zittrain, J. L. (2018). The science of fake news. Science, 359(6380), 1094–1096. https://doi.org/10.1126/science.aao2998
LekshmiAmmal, H. R., & Madasamy, A. K. (2025). A reasoning based explainable multimodal fake news detection for low resource language using large language models and transformers. Journal of Big Data, 12(1), 46. https://doi.org/10.1186/s40537-025-01093-x
Lin, S.-Y., Chen, Y.-C., Chang, Y.-H., Lo, S.-H., & Chao, K.-M. (2024). Text–image multimodal fusion model for enhanced fake news detection. Science Progress, 107(4), 00368504241292685. https://doi.org/10.1177/00368504241292685
Lundberg, S., & Lee, S.-I. (2017). A Unified Approach to Interpreting Model Predictions (arXiv:1705.07874). arXiv. https://doi.org/10.48550/arXiv.1705.07874
Luttrell, R., Davis, J., & Welch, C. (2025). Source attribution and detection strategies for AI-era journalism. Telecommunications Policy, 49(10), 103053. https://doi.org/10.1016/j.telpol.2025.103053
Mishima, K., & Yamana, H. (2022). A Survey on Explainable Fake News Detection. IEICE Transactions on Information and Systems, E105.D(7), 1249–1257. https://doi.org/10.1587/transinf.2021EDR0003
Moyo, B. V., Tuyikeze, T., Matsebula, F., & Obagbuwa, I. C. (2026). An AI-driven conceptual framework for detecting fake news and deepfake content: A systematic review. Frontiers in Artificial Intelligence, 9. https://doi.org/10.3389/frai.2026.1737790
Muñoz, S., & Iglesias, C. Á. (2024). Exploiting Content Characteristics for Explainable Detection of Fake News. Big Data and Cognitive Computing, 8(10), 129. https://doi.org/10.3390/bdcc8100129
Nasir, J. A., Khan, O. S., & Varlamis, I. (2021). Fake news detection: A hybrid CNN-RNN based deep learning approach. International Journal of Information Management Data Insights, 1(1), 100007. https://doi.org/10.1016/j.jjimei.2020.100007
Nasser, M., Arshad, N. I., Ali, A., Alhussian, H., Saeed, F., Da’u, A., & Nafea, I. (2025). A systematic review of multimodal fake news detection on social media using deep learning models. Results in Engineering, 26, 104752. https://doi.org/10.1016/j.rineng.2025.104752
Nwaiwu, S., Jongsawat, N., & Tungkasthan, A. (2025). Decoding Disinformation: A Feature-Driven Explainable AI Approach to Multi-Domain Fake News Detection. Applied Sciences, 15(17), 9498. https://doi.org/10.3390/app15179498
Ognyanova, K., Lazer, D., Robertson, R. E., & Wilson, C. (2020). Misinformation in action: Fake news exposure is linked to lower trust in media, higher trust in government when your side is in power. Harvard Kennedy School Misinformation Review, 1(3). https://doi.org/10.37016/mr-2020-024
Palla, Z., & Kostarella, I. (2025). Journalists’ Perspectives on the Role of Artificial Intelligence in Enhancing Quality Journalism in Greek Local Media. Societies, 15(4), 89. https://doi.org/10.3390/soc15040089
Pennycook, G., & Rand, D. G. (2021). The Psychology of Fake News. Trends in Cognitive Sciences, 25(5), 388–402. https://doi.org/10.1016/j.tics.2021.02.007
Rai, N., Kumar, D., Kaushik, N., Raj, C., & Ali, A. (2022). Fake News Classification using transformer based enhanced LSTM and BERT. International Journal of Cognitive Computing in Engineering, 3, 98–105. https://doi.org/10.1016/j.ijcce.2022.03.003
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, 1135–1144. https://doi.org/10.1145/2939672.2939778
Sharma, U., & Singh, J. (2024). A comprehensive overview of fake news detection on social networks. Social Network Analysis and Mining, 14(1), 120. https://doi.org/10.1007/s13278-024-01280-3
Shu, K., Mahudeswaran, D., Wang, S., Lee, D., & Liu, H. (2020). FakeNewsNet: A Data Repository with News Content, Social Context, and Spatiotemporal Information for Studying Fake News on Social Media. Big Data, 8(3), 171–188. https://doi.org/10.1089/big.2020.0062
Sundararajan, M., Taly, A., & Yan, Q. (2017). Axiomatic Attribution for Deep Networks. Proceedings of the 34th International Conference on Machine Learning, 3319–3328. https://proceedings.mlr.press/v70/sundararajan17a.html
Tandoc Jr., E. C., Lim, Z. W., & Ling, R. (2018). Defining “Fake News”: A typology of scholarly definitions. Digital Journalism, 6(2), 137–153. https://doi.org/10.1080/21670811.2017.1360143
Vallayil, M., Nand, P., Yan, W. Q., & Allende-Cid, H. (2023). Explainability of Automated Fact Verification Systems: A Comprehensive Review. Applied Sciences, 13(23), 12608. https://doi.org/10.3390/app132312608
Wang, W. Y. (2017). “Liar, Liar Pants on Fire”: A New Benchmark Dataset for Fake News Detection. In R. Barzilay & M.-Y. Kan (Eds.), Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (pp. 422–426). Association for Computational Linguistics. https://doi.org/10.18653/v1/P17-2067
Wang, X., Cabrio, E., & Villata, S. (2025). When automated fact-checking meets argumentation: Unveiling fake news through argumentative evidence. Argument & Computation, 16(3), 405–424. https://doi.org/10.1177/19462174251330980
Zhou, X., & Zafarani, R. (2020). A Survey of Fake News: Fundamental Theories, Detection Methods, and Opportunities. ACM Computing Surveys (CSUR), 53(5), 109:1-109:40. https://doi.org/10.1145/3395046
Zubiaga, A., Aker, A., Bontcheva, K., Liakata, M., & Procter, R. (2018). Detection and Resolution of Rumours in Social Media: A Survey. ACM Computing Surveys (CSUR), 51(2), 32:1-32:36. https://doi.org/10.1145/3161603
Downloads
Published
Issue
Section
License
Copyright (c) 2026 K. M. Tousif Bin Parves, Prashanta Kumar Shill
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright Notice
We declare that:
1. This paper has not been published in the same form elsewhere.
2. It will not be submitted anywhere else for publication prior to acceptance/rejection by this Journal.
3. A copyright permission is obtained for materials published elsewhere and which require this permission for reproduction.
The corresponding author signs for and accepts responsibility for releasing this material on behalf of any and all co-authors. This agreement is to be signed by at least one of the authors who have obtained the assent of the co-author(s) where applicable.
Retained Rights/Terms and Conditions
1. Authors retain all proprietary rights in any process, procedure, or article of manufacture described in the Work.
2. Authors may reproduce or authorize others to reproduce the work or derivative works for the authors personal use or for company use.
3. Although authors are permitted to re-use all or portions of the Work in other works, this does not include granting third-party requests for reprinting, republishing, or other types of re-use.
