Comparative Analysis of CatBoost, Random Forest, and XGBoost for Predicting Student Academic Performance
DOI:
https://doi.org/10.37278/sisinfo.v8i2.1572Keywords:
Student Academic Performance, CatBoost, Random Forest, XGBoost, Educational Data MiningAbstract
Predicting student academic performance has become an important topic in educational data mining because it enables educational institutions to identify students who may require academic support at an early stage. This study compares the performance of three ensemble learning algorithms—CatBoost, Random Forest, and XGBoost—in predicting students' final academic grades using the Student Performance Dataset obtained from Kaggle. The dataset contains 649 student records with demographic, family, behavioral, and academic information. Before model development, the data were preprocessed through categorical feature encoding and feature engineering, resulting in five additional variables: parent_education, previous_grade, total_alcohol, attendance_category, and study_efficiency. The dataset was divided into training and testing sets using an 80:20 ratio. Hyperparameter optimization was performed only for CatBoost using the Optuna framework with 20 optimization trials, while Random Forest and XGBoost were trained using predefined parameter settings. Model performance was assessed using Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and the Coefficient of Determination (R²). The results indicate that CatBoost achieved the best overall performance, with an RMSE of 1.2888 and an R² of 0.8297, outperforming the other two algorithms. Feature importance analysis also revealed that G2 and previous_grade were the strongest predictors of students' final academic performance. These results suggest that CatBoost is a reliable approach for student performance prediction and can support data-driven academic decision-making.
References
M. Yağcı, “Educational data mining: prediction of students’ academic performance using machine learning algorithms,” Smart Learn. Environ., vol. 9, no. 1, p. 11, 2022, doi: 10.1186/s40561-022-00192-z.
Syed Eirfan Atthar, “International Conference on Emerging Markets (ICEM) 2026 AI-Enabled Digital Twins for Low-Carbon Logistics in Emerging Markets: A Human-Centric Framework for Cold-Chain Energy Efficiency and CBAM-Ready Supply Chains in India ‘A Human-Centric Framework for Energy-Efficient and Sustainable Cold-Chain Supply Chains,’” Int. J. Res. Innov. Soc. Sci., vol. 10, no. 19, pp. 41–57, 2026, doi: 10.47772/IJRISS.
Murnawan, S. Lestari, R. Samihardjo, and D. A. Dewi, “Sustainable Educational Data Mining Studies: Identifying Key Factors and Techniques for Predicting Student Academic Performance,” J. Appl. Data Sci., vol. 5, no. 3, pp. 1325–1342, 2024, doi: 10.47738/jads.v5i3.347.
M. Bilal, M. Omar, W. Anwar, R. H. Bokhari, and G. S. Choi, “The role of demographic and academic features in a student performance prediction,” Sci. Rep., vol. 12, no. 1, p. 12508, 2022, doi: 10.1038/s41598-022-15880-6.
H. ŞEVGİN, “A comparative study of ensemble methods in the field of education: Bagging and Boosting algorithms,” Int. J. Assess. Tools Educ., vol. 10, no. 3, pp. 544–562, 2023, doi: 10.21449/ijate.1167705.
A. Villar and C. R. V. de Andrade, “Supervised machine learning algorithms for predicting student dropout and academic success: a comparative study,” Discov. Artif. Intell., vol. 4, no. 1, p. 2, 2024, doi: 10.1007/s44163-023-00079-z.
R. C. Lim and H. Harvianto, “Comparative Analysis of Decision Tree, Random Forest, and XGBoost for Student Category Prediction,” Int. J. Comput. Sci. Humanit. AI, vol. 3, no. 1, p. 9, 2026, doi: 10.21512/ijcshai.v3i1.14936.
C. Chaka, “Educational data mining, student academic performance prediction, prediction methods, algorithms and tools: an overview of reviews,” J. E-Learning Knowl. Soc., vol. 18, no. 2, pp. 58–69, 2022, doi: 10.20368/1971-8829/1135578.
Z. Alamgir, H. Akram, S. Karim, and A. Wali, “Enhancing Student Performance Prediction via Educational Data Mining on Academic Data,” Informatics Educ., vol. 23, no. 1, pp. 1–24, 2024, doi: 10.15388/infedu.2024.04.
W. Xiao, P. Ji, and J. Hu, “A survey on educational data mining methods used for predicting students’ performance,” Eng. Reports, vol. 4, no. 5, p. e12482, May 2022, doi: https://doi.org/10.1002/eng2.12482.
M. L. Murnawan Sri; Samihardjo, Rosalim; Dewi, Deshinta Arrova, “Sustainable Educational Data Mining Studies: Identifying Key Factors and Techniques for Predicting Student Academic Performance,” J. Appl. Data Sci., no. Vol 5, No 3: SEPTEMBER 2024, pp. 1325–1342, 2024, [Online]. Available: http://bright-journal.org/Journal/index.php/JADS/article/view/347/233
M. N. Gul, W. Abbasi, M. Z. Babar, A. Aljohani, and M. Arif, “Data driven decisions in education using a comprehensive machine learning framework for student performance prediction,” Discov. Comput., vol. 28, no. 1, p. 153, 2025, doi: 10.1007/s10791-025-09585-3.
N. Rohani, B. Rohani, and A. Manataki, “ClickTree: A Tree-based Method for Predicting Math Students’ Performance Based on Clickstream Data,” J. Educ. Data Min., vol. 16, no. 2, pp. 32–57, 2024, doi: 10.5281/zenodo.13627655.
W. Cui, A. Sangsongfar, and N. Amdee, “A Comparative Study of the Applicability of Regression Models in Predicting Student Academic Performance,” Naresuan Univ. Eng. J., vol. 19, no. 1, pp. 39–49, 2024.
D. Ying and J. Ma, “Student Performance Prediction with Regression Approach and Data Generation,” Appl. Sci., vol. 14, no. 3, 2024, doi: 10.3390/app14031148.
M. Kumar, N. Singh, J. Wadhwa, P. Singh, G. Kumar, and A. Qtaishat, “Utilizing Random Forest and XGBoost Data Mining Algorithms for Anticipating Students’ Academic Performance,” 2023.
M. Kinter and N. E. Sherman, “Utilizing Random Forest and XGBoost Data Mining Algorithms for Anticipating Students’ Academic Performance.”
D. N. Muhammady, H. A. E. Nugraha, V. R. S. Nastiti, and C. S. K. Aditya, “Students Final Academic Score Prediction Using Boosting Regression Algorithms,” J. Ilm. Tek. Elektro Komput. dan Inform., vol. 10, no. 1, p. 154, 2024, doi: 10.26555/jiteki.v10i1.28352.
Z. Fan, J. Gou, and S. Weng, “Complementary CatBoost based on residual error for student performance prediction,” Pattern Recognit., vol. 161, p. 111265, 2025, doi: https://doi.org/10.1016/j.patcog.2024.111265.
M. N. Gul, W. Abbasi, M. Z. Babar, A. Aljohani, and M. Arif, Data driven decisions in education using a comprehensive machine learning framework for student performance prediction, vol. 28, no. 1. Springer Netherlands, 2025. doi: 10.1007/s10791-025-09585-3.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Yayu Nur Faidah, Agung Rachmat Raharja, Andri Feriansyah, Ahmad Labudi
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish articles in SisInfo : Jurnal Sistem Informasi dan Informatika agree to the following terms:
- Authors retain copyright of the article and grant the journal right of first publication with the work simultaneously licensed under a CC-BY-SA or The Creative Commons Attribution-ShareAlike License.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
