An Explainable CatBoost Framework Optimized with Optuna for Predicting Students' Mathematics Proficiency
DOI:
https://doi.org/10.37278/sisinfo.v8i2.1575Keywords:
CatBoost, Optuna, SHAP, Explainable Artificial IntelligenceAbstract
Predicting students' mathematics proficiency is an important task in educational data mining because it supports educational assessment and data-informed decision making. This study proposes an explainable machine learning framework by integrating CatBoost, Optuna, and SHAP to classify students into different levels of mathematics proficiency. The study uses the Student Performance in Mathematics dataset from Kaggle, which contains 1,000 student records with demographic, socioeconomic, and academic characteristics. Mathematics scores were grouped into three proficiency categories (Low, Medium, and High), transforming the problem into a multiclass classification task. The dataset was divided into training and testing subsets using an 80:20 stratified split. Hyperparameter optimization was performed on the training data using Optuna with 30 optimization trials and five-fold stratified cross-validation, while the testing dataset was reserved exclusively for the final evaluation. The optimized CatBoost model was compared with XGBoost and LightGBM under identical experimental settings. Performance was evaluated using Accuracy, Precision, Recall, F1-score, ROC-AUC, and Confusion Matrix. The optimized CatBoost model achieved the highest observed performance among the evaluated models, with an accuracy of 80.00%, a weighted F1-score of 80.07%, and a ROC-AUC of 0.9190. SHAP analysis identified reading score, writing score, and gender as the most influential predictors of mathematics proficiency. The results indicate that the proposed framework provides competitive classification performance and model interpretability within the studied dataset.
References
C. Romero and S. Ventura, “Educational data mining and learning analytics: An updated survey,” Wiley Interdiscip. Rev. Data Min. Knowl. Discov., vol. 10, no. 3, 2020, doi: 10.1002/widm.1355.
Q. Xie et al., “Cognitive style and Students’ academic achievement: a meta-analysis,” Front. Educ. (Lausanne)., vol. 10, no. October, 2025, doi: 10.3389/feduc.2025.1634732.
M. Wang, M. E. E. Mohd Matore, and R. Rosli, “A systematic literature review on analytical thinking development in mathematics education: trends across time and countries,” Front. Psychol., vol. 16, no. June, pp. 1–10, 2025, doi: 10.3389/fpsyg.2025.1523836.
W. Xiao, P. Ji, and J. Hu, “A survey on educational data mining methods used for predicting students’ performance,” Engineering Reports, vol. 4, no. 5, p. e12482, May 2022, doi: https://doi.org/10.1002/eng2.12482.
M. Yağcı, “Educational data mining: prediction of students’ academic performance using machine learning algorithms,” Smart Learning Environments, vol. 9, no. 1, p. 11, 2022, doi: 10.1186/s40561-022-00192-z.
A. Villar and C. R. V. de Andrade, “Supervised machine learning algorithms for predicting student dropout and academic success: a comparative study,” Discover Artificial Intelligence, vol. 4, no. 1, p. 2, 2024, doi: 10.1007/s44163-023-00079-z.
A. Abukader, A. Alzubi, and O. R. Adegboye, “Intelligent System for Student Performance Prediction: An Educational Data Mining Approach Using Metaheuristic-Optimized LightGBM with SHAP-Based Learning Analytics,” Applied Sciences (Switzerland), vol. 15, no. 20, pp. 1–25, 2025, doi: 10.3390/app152010875.
L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, and A. Gulin, “Catboost: Unbiased boosting with categorical features,” Adv. Neural Inf. Process. Syst., vol. 2018-Decem, no. Section 4, pp. 6638–6648, 2018.
M. Hassanali, M. Soltanaghaei, T. Javdani Gandomani, and F. Zamani Boroujeni, “Software development effort estimation using boosting algorithms and automatic tuning of hyperparameters with Optuna,” Journal of Software: Evolution and Process, vol. 36, no. 9, p. e2665, 2024, doi: https://doi.org/10.1002/smr.2665.
V. Hassija et al., “Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence,” Cognit. Comput., vol. 16, no. 1, pp. 45–74, 2024, doi: 10.1007/s12559-023-10179-8.
Md. T. Hosain, J. R. Jim, M. F. Mridha, and M. M. Kabir, “Explainable AI approaches in deep learning: Advancements, applications and challenges,” Computers and Electrical Engineering, vol. 117, p. 109246, 2024, doi: https://doi.org/10.1016/j.compeleceng.2024.109246.
G. Türkmen, “The Review of Studies on Explainable Artificial Intelligence in Educational Research,” Journal of Educational Computing Research, vol. 63, no. 2, pp. 277–310, 2025, doi: 10.1177/07356331241310915.
E. Ben George, “Explainable AI Methods for Predicting Student Grades and Improving Academic Success,” Journal of Information Systems Engineering and Management, vol. 10, no. 23s, pp. 117–126, 2025, doi: 10.52783/jisem.v10i23s.3680.
J. T. Hancock and T. M. Khoshgoftaar, “CatBoost for big data: an interdisciplinary review,” J. Big Data, vol. 7, no. 1, p. 94, 2020, doi: 10.1186/s40537-020-00369-8.
B. Bischl et al., “Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges,” WIREs Data Mining and Knowledge Discovery, vol. 13, no. 2, p. e1484, Mar. 2023, doi: https://doi.org/10.1002/widm.1484.
J. M. Tan et al., “Hyperparameter optimization: Classics, acceleration, online, multi-objective, and tools,” Mathematical Biosciences and Engineering, vol. 21, no. 6, pp. 6289–6335, 2024, doi: 10.3934/mbe.2024275.
H. Almarzooq and U. bin Waheed, “Automating hyperparameter optimization in geophysics with Optuna: A comparative study,” Geophys. Prospect., vol. 72, no. 5, pp. 1778–1788, Jun. 2024, doi: https://doi.org/10.1111/1365-2478.13484.
M. Mersha, K. Lam, J. Wood, A. K. AlShami, and J. Kalita, “Explainable artificial intelligence: A survey of needs, techniques, applications, and future direction,” Neurocomputing, vol. 599, 2024, doi: 10.1016/j.neucom.2024.128111.
M. Saarela and V. Podgorelec, “Recent Applications of Explainable AI (XAI): A Systematic Literature Review,” Applied Sciences (Switzerland), vol. 14, no. 19, 2024, doi: 10.3390/app14198884.
L. C. Nnadi, Y. Watanobe, M. Rahman, and A. M. John-otumu, “Scopus - Document details - Prediction of Students’ Adaptability Using Explainable AI in Educational Machine Learning Models,” 2024.
Z. Ersozlu, S. Taheri, and I. Koch, “A review of machine learning methods used for educational data,” Educ. Inf. Technol. (Dordr)., vol. 29, no. 16, pp. 22125–22145, 2024, doi: 10.1007/s10639-024-12704-0.
Z. Fan, J. Gou, and S. Weng, “Complementary CatBoost based on residual error for student performance prediction,” Pattern Recognit., vol. 161, p. 111265, 2025, doi: https://doi.org/10.1016/j.patcog.2024.111265.
A. Almalawi, B. Soh, A. Li, and H. Samra, “Predictive Models for Educational Purposes: A Systematic Review,” Big Data and Cognitive Computing, vol. 8, no. 12, pp. 1–42, 2024, doi: 10.3390/bdcc8120187.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Andri Feriansyah, Ahmad Labudi, Yayu Nur Faidah, Agung Rachmat
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish articles in SisInfo : Jurnal Sistem Informasi dan Informatika agree to the following terms:
- Authors retain copyright of the article and grant the journal right of first publication with the work simultaneously licensed under a CC-BY-SA or The Creative Commons Attribution-ShareAlike License.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
