AI-related measures and self-reported lesson-aim attainment: an explainable machine-learning analysis of TALIS 2024

Vollständiger Abstract

Worum geht es in dieser Arbeit?

Introduction Artificial intelligence (AI) is increasingly used to support teachers' lesson planning, feedback, assessment, and learning-material adaptation in school settings. Effective teaching practices may also support health literacy, social-emotional development, and health-promoting learning environments, although TALIS does not directly measure public-health education outcomes. Existing studies often treat AI use as a simple adoption variable and provide limited cross-national evidence on whether AI-related measures provide incremental predictive information after accounting for teacher background, teachers' work-related stress, and education-system context. Methods A nested explainable machine-learning framework was developed using TALIS 2024 teacher data to test the incremental predictive information supplied by AI-related measures. The participants were 88,991 primary, lower-secondary, and upper-secondary teachers (ISCED 1–3) from 55 countries/economies who were eligible for the AI-module analysis and had a valid outcome. The seven-item outcome captured teachers' reports of how far lessons taught in the previous week achieved specified instructional and classroom-management aims; a relatively high indicator denoted scores at or above the 75th percentile within the same country/economy and ISCED stratum. Matched baseline and AI-added regularized LightGBM models were compared using country-by-school grouped validation and early stopping. Threshold, continuous-outcome, survey-weighted-evaluation, missingness, and country-heterogeneity sensitivity analyses were conducted. Results In the final matched analysis, the AI-added model increased independent-test AUROC from 0.6021 to 0.6299 and AUPRC from 0.4138 to 0.4476, while reducing the Brier score from 0.2114 to 0.2071 and log loss from 0.6126 to 0.6029. Paired bootstrap intervals indicated statistically stable, but practically modest, gains (observed ΔAUROC = 0.0277, 95% CI 0.0215–0.0334; observed ΔAUPRC = 0.0338, 95% CI 0.0262–0.0411). AUROC gains were similar at the 70th and 80th percentile thresholds, and the continuous-outcome R 2 increased from 0.1745 to 0.1911. Country-specific AUROC gains were positive in 48 of 55 countries/economies but varied substantially. Discussion AI-related perceptions and breadth of task use supplied modest incremental predictive information for predicting relatively high self-reported lesson-aim attainment. “Relatively high” denotes only a within-country/economy and ISCED ranking and does not indicate objectively superior teaching or better student learning. The modest discrimination, cross-sectional design, routed missingness, and self-reported outcome preclude causal interpretation and use for teacher screening or other high-stakes individual decisions. Public-health relevance is presented as a potential extension to future research in health-promoting school settings, not as a directly measured outcome.