Educational Data Mining of Student Learning Gains: Feature Relevance and Limits of Prediction

Abstract

Educational Data Mining (EDM) is widely used to predict achievement, yet less is known about its ability to predict learning gain. This study examined whether previous academic achievement, prior knowledge, study time, and pedagogical intervention were associated with and predictive of learning gain among 120 secondary school students participating in Guided Inquiry-Based Learning (GIBL) or Problem-Based Learning (PBL). Gain Score was calculated as post-test minus pre-test. A two-way ANOVA tested intervention and prior-knowledge effects, while Linear Regression, Support Vector Machine, k-Nearest Neighbors, Random Forest, and Decision Tree models were evaluated with 10-fold cross-validation. RReliefF assessed predictor relevance and RMSE, MAE, and R² assessed model performance. The ANOVA showed no significant effects of intervention, prior knowledge, or their interaction. Previous Grade was the most relevant predictor, followed by Study Hour, Pedagogical Intervention, and Prior Knowledge. Linear Regression yielded the lowest errors (RMSE = 2.666; MAE = 2.130), but all models had negative R² values. The findings show that relative predictor relevance and model superiority do not necessarily indicate adequate individual-level prediction; richer process-level learning data are needed.