Interpretable Machine Learning for Early Detection of Academically At-Risk Students Using Behavioral, Socio-Demographic and Learning-Related Factors

Vadlya Maarif, Eka Rahmawati, Candra Agustina

Abstract


The increasing adoption of learning analytics in higher education has encouraged the development of machine learning models for the early detection of academically at-risk students. However, many predictive models emphasize accuracy while providing limited interpretability for educators and academic advisors. This study proposes an interpretable machine learning framework for predicting academic risk using behavioral, socio-demographic, and learning-related student variables. A publicly available synthetic Kaggle dataset consisting of 500 student records was used as a controlled dataset for methodological validation. The data were preprocessed through missing-value handling, standardization, and one-hot encoding before being divided into training and testing sets. Several classifiers were evaluated, including Naive Bayes, Support Vector Machine, Random Forest, Logistic Regression, and XGBoost. Logistic Regression was employed as an interpretable baseline model, while XGBoost was used as a comparative ensemble classifier. Model performance was evaluated using accuracy, precision, recall, F1-score, AUC, and confusion matrix analysis. The results show that Logistic Regression achieved the highest accuracy and F1-score, with an accuracy of 0.8600 and an F1-score of 0.7812. XGBoost achieved the highest AUC value of 0.9278, followed closely by Logistic Regression with an AUC of 0.9246. Random Forest and XGBoost produced the lowest false negative values, indicating their ability to identify at-risk students more effectively. SHAP-based explainability revealed that weekly study hours, assignment completion, class attendance, and sleep duration were the most influential predictors. These findings suggest that interpretable machine learning can support academic early-warning systems by providing transparent prediction results. Since the dataset is synthetic, the findings should be interpreted as methodological validation rather than direct generalization to real student populations.

Keywords


Academic Risk; Explainable Artificial Intelligence Interpretable Machine Learning; Learning Analytics Synthetic Educational Data

Full Text:

Link Download

References


Abdrakhmanov, R., Zhaxanova, A., Karatayeva, M., Zholaushievna Niyazova, G., Berkimbayev, K., Tuimebayev, A., Kazakhstan University named after MAuezov, S., Auezov South Kazakhstan University, M., & Akhmet Yassawi, K. (2024). Development of a Framework for Predicting Students’ Academic Performance in STEM Education using Machine Learning Methods. In IJACSA) International Journal of Advanced Computer Science and Applications (Vol. 15, Number 1). www.ijacsa.thesai.org

Ahmed, A., Nipa, F. A., Bhuyian, W. U., Mushfique, K. M., Shahin, K. I., Nguyen, H. H., & Farid, D. M. (2024). Students’ performance prediction employing Decision Tree. CTU Journal of Innovation and Sustainable Development, 16(Special Issue), 42–51. https://doi.org/10.22144/ctujoisd.2024.321

Almarzouki, A. F. (2024). Stress, working memory, and academic performance: a neuroscience perspective. In Stress (Vol. 27, Number 1). Taylor and Francis Ltd. https://doi.org/10.1080/10253890.2024.2364333

Bakhtiar, A., & Hadwin, A. F. (2022). Motivation From a Self-Regulated Learning Perspective: Application to School Psychology. Canadian Journal of School Psychology, 37(1), 93–116. https://doi.org/10.1177/08295735211054699

Cai, S., Jin, Y., Lan, X., & Liu, Y. (2024). The cognitive psychology of learning anxiety: Effects on academic performance. In Addressing Global Challenges - Exploring Socio-Cultural Dynamics and Sustainable Solutions in a Changing World: Proceedings of International Symposium on Humanities (pp. 929–935). CRC Press. https://doi.org/10.1201/9781032676043-130

Cao, X. (2023). The application of structural equation model in psychological research. CNS Spectrums, 28(S1), S17–S19. https://doi.org/10.1017/s1092852923000858

Chen, Y., & Zhai, L. (2023). A comparative study on student performance prediction using machine learning. Education and Information Technologies, 28(9), 12039–12057. https://doi.org/10.1007/s10639-023-11672-1

Cheng, B., Liu, Y., & Jia, Y. (2024). Evaluation of students’ performance during the academic period using the XG-Boost Classifier-Enhanced AEO hybrid model. Expert Systems with Applications, 238. https://doi.org/10.1016/j.eswa.2023.122136

Deng, L., & Yuan, K. H. (2023). Which method is more powerful in testing the relationship of theoretical constructs? A meta comparison of structural equation modeling and path analysis with weighted composites. Behavior Research Methods, 55(3), 1460–1479. https://doi.org/10.3758/s13428-022-01838-z

Gaftandzhieva, S., Hussain, S., Hilčenko, S., Doneva, R., & Boykova, K. (2023). Data-driven Decision Making in Higher Education Institutions: State-of-play. IJACSA) International Journal of Advanced Computer Science and Applications, 14(6), 2023. www.ijacsa.thesai.org

Gligorea, I., Yaseen, M. U., Cioca, M., Gorski, H., & Oancea, R. (2022). An Interpretable Framework for an Efficient Analysis of Students’ Academic Performance. Sustainability (Switzerland), 14(14). https://doi.org/10.3390/su14148885

Guevara-Reyes, R., Ortiz-Garcés, I., Andrade, R., Cox-Riquetti, F., & Villegas-Ch, W. (2025). Machine learning models for academic performance prediction: interpretability and application in educational decision-making. Frontiers in Education, 10. https://doi.org/10.3389/feduc.2025.1632315

Han, F., Vaculíková, J., & Juklová, K. (2022). The relations between Czech undergraduates’ motivation and emotion in self-regulated learning, learning engagement, and academic success in blended course designs: Consistency between theory-driven and data-driven approaches. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.1001202

Jiao, Y., Bai, B., & Li, J. (2025). Profiling EFL Students’ Motivation and Anxiety in Collaborative Learning: Associations With Teachers’ Support, Self-Regulated Learning, and Achievement. International Journal of Applied Linguistics (United Kingdom). https://doi.org/10.1111/ijal.12831

Kassaw, C., & Demareva, V. (2025). Psychosocial and cognitive predictors of academic achievement among higher education students in Southern Ethiopia. PLOS ONE, 20(11 November). https://doi.org/10.1371/journal.pone.0335968

Li, Q., & Zhou, J. (2025). A Comparative Analysis of Extreme Gradient Boosting, Decision Tree, Support Vector Machines, and Random Forest Algorithm in Data Analysis of College Students’ Psychological Health. Informatica (Slovenia), 49(15), 127–134. https://doi.org/10.31449/inf.v49i15.7004

Lynam, S., Cachia, M., & Stock, R. (2024). An evaluation of the factors that influence academic success as defined by engaged students. Educational Review, 76(3), 586–604. https://doi.org/10.1080/00131911.2022.2052808

Malik, S., Patro, S. G. K., Mahanty, C., Hegde, R., Naveed, Q. N., Lasisi, A., Buradi, A., Emma, A. F., & Kraiem, N. (2025). Advancing educational data mining for enhanced student performance prediction: a fusion of feature selection algorithms and classification techniques with dynamic feature ensemble evolution. Scientific Reports, 15(1). https://doi.org/10.1038/s41598-025-92324-x

Mao, H., Khanal, R., Qu, C. Z., Kong, H. F., & Jiang, T. Y. (2025). What factors enhance students’ achievement? A machine learning and interpretable methods approach. PLoS ONE, 20(5 May). https://doi.org/10.1371/journal.pone.0323345

Matz, S. C., Bukow, C. S., Peters, H., Deacons, C., & Stachl, C. (2023). Using machine learning to predict student retention from socio-demographic characteristics and app-based engagement metrics. Scientific Reports, 13(1). https://doi.org/10.1038/s41598-023-32484-w

Mehrad, A., Bouzedif, M., & Rweramila, N. G. (2024). Integrating Psychology, Cognitive, and Behavioral Science in the Concept of Education and Student Success in the Educational System. In Tuijin Jishu/Journal of Propulsion Technology (Vol. 45, Number 3).

Muraina, I. O., Aiyegbusi, E., & Abam, S. (2023). Decision Tree Algorithm Use in Predicting Students’ Academic Performance in Advanced Programming Course. International Journal of Higher Education Pedagogies, 3(4), 13–23. https://doi.org/10.33422/ijhep.v3i4.274

Musa, A. B. (2024). Understanding Student Performance in Foundation Year: Insights from Logistic Regression, Naïve Bayes, and Random Forest Models. International Journal of Information and Education Technology, 14(12), 1716–1723. https://doi.org/10.18178/ijiet.2024.14.12.2202

Nachouki, M., Mohamed, E. A., Mehdi, R., & Abou Naaj, M. (2023). Student course grade prediction using the random forest algorithm: Analysis of predictors’ importance. Trends in Neuroscience and Education, 33. https://doi.org/10.1016/j.tine.2023.100214

Navarro, R., Vega, V., Bayona, H., Bernal, V., & Garcia, A. (2023). Relationship between technology acceptance model, self-regulation strategies, and academic self-efficacy with academic performance and perceived learning among college students during remote education. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1227956

Nnadi, L. C., Watanobe, Y., Rahman, M. M., & John-Otumu, A. M. (2024). Prediction of Students’ Adaptability Using Explainable AI in Educational Machine Learning Models. Applied Sciences (Switzerland), 14(12). https://doi.org/10.3390/app14125141

Pelima, L. R., Sukmana, Y., & Rosmansyah, Y. (2024). Predicting University Student Graduation Using Academic Performance and Machine Learning: A Systematic Literature Review. IEEE Access, 12, 23451–23465. https://doi.org/10.1109/ACCESS.2024.3361479

Rosseel, Y., & Loh, W. W. (2022). A Structural After Measurement Approach to Structural Equation Modeling. Psychological Methods, 29(3), 561–588. https://doi.org/10.1037/met0000503

Schroeder, M. N., & Murphy, J. A. (2025). Blending Early Warning and Remediation Processes to Facilitate Student Success. American Journal of Pharmaceutical Education, 89(7). https://doi.org/10.1016/j.ajpe.2025.101420

Sevvanthi, K., Ganapathy, S., Penumadu, P., & Harichandrakumar, K. T. (2023). Comparing the predictive performance of a decision tree with logistic regression for oral cavity cancer mortality: A retrospective study. Cancer Research, Statistics, and Treatment, 6(1), 103–110. https://doi.org/10.4103/crst.crst_234_22

Singh, H., Kaur, B., Sharma, A., & Singh, A. (2024). Framework for suggesting corrective actions to help students intended at risk of low performance based on experimental study of college students using explainable machine learning model. Education and Information Technologies, 29(7), 7997–8034. https://doi.org/10.1007/s10639-023-12072-1

Singh, H. P., & Alhulail, H. N. (2022). Predicting Student-Teachers Dropout Risk and Early Identification: A Four-Step Logistic Regression Approach. IEEE Access, 10, 6470–6482. https://doi.org/10.1109/ACCESS.2022.3141992

Susnjak, T., Ramaswami, G. S., & Mathrani, A. (2022). Learning analytics dashboard: a tool for providing actionable insights to learners. International Journal of Educational Technology in Higher Education, 19(1). https://doi.org/10.1186/s41239-021-00313-7

Syed Mustapha, S. M. F. D. (2023). Predictive Analysis of Students’ Learning Performance Using Data Mining Techniques: A Comparative Study of Feature Selection Methods. Applied System Innovation, 6(5). https://doi.org/10.3390/asi6050086

Tang, B., Li, S., & Zhao, C. (2024). Predicting the Performance of Students Using Deep Ensemble Learning. Journal of Intelligence, 12(12). https://doi.org/10.3390/jintelligence12120124

Tao, T., Sun, C., Wu, Z., Yang, J., & Wang, J. (2022). Deep Neural Network-Based Prediction and Early Warning of Student Grades and Recommendations for Similar Learning Approaches. Applied Sciences (Switzerland), 12(15). https://doi.org/10.3390/app12157733

Wang, S., & Luo, B. (2024). Academic achievement prediction in higher education through interpretable modeling. PLoS ONE, 19(9 September). https://doi.org/10.1371/journal.pone.0309838

Wen, X., & Juan, H. (2023). Early Prediction of Students’ Performance Using a Deep Neural Network Based on Online Learning Activity Sequence. Applied Sciences (Switzerland), 13(15). https://doi.org/10.3390/app13158933

Yin, X. (2025). Digital Transformation of Higher Education: Trends, Innovation Models and Strategic Paths. Pakistan Journal of Life and Social Sciences (PJLSS), 23(1). https://doi.org/10.57239/pjlss-2025-23.1.00100

Yu, J., Huang, C., He, T., Wang, X., & Zhang, L. (2022). Investigating students’ emotional self-efficacy profiles and their relations to self-regulation, motivation, and academic performance in online learning contexts: A person-centered approach. Education and Information Technologies, 27(8), 11715–11740. https://doi.org/10.1007/s10639-022-11099-0




DOI: http://dx.doi.org/10.35671/telematika.v19i2.3318

Refbacks

  • There are currently no refbacks.


 


Indexed by:

   

Telematika
ISSN: 2442-4528 (online) | ISSN: 1979-925X (print)
Published by : Universitas Amikom Purwokerto
Jl. Let. Jend. POL SUMARTO Watumas, Purwonegoro - Purwokerto, Indonesia


Creative Commons License This work is licensed under a Creative Commons Attribution 4.0 International License .