AUTHOR=Reinhold Lhea , Händel Marion , Naujoks-Schober Nick 

TITLE=AI-teacher agreement in evaluating learning diaries

JOURNAL=Frontiers in Education

VOLUME=Volume 10 - 2025

YEAR=2025

URL=https://www.frontiersin.org/journals/education/articles/10.3389/feduc.2025.1601789

DOI=10.3389/feduc.2025.1601789

ISSN=2504-284X

ABSTRACT=Learning diaries are reflective tools, often used as formative assessments in adult education with the aim to promote cognitive and metacognitive learning strategies. As grading of and feedback on learning diaries is effortful for teachers, artificial intelligence (AI) may assist teachers in evaluating learning diaries. A prerequisite is that AI's ratings show high accordance with the teachers' ratings. AI accuracy, measured via absolute accuracy and bias, is the focus of the current study with N = 540 learning diary entries focusing on learning strategies, seven teachers, and ChatGPT-4o. Findings revealed that AI evaluations align closely with teacher assessments, indicated by high overall accuracy and low bias. Interestingly, the accuracy varied based on the types of learning strategies assessed in the diaries. Additionally, individual teacher assessments influenced the alignment between human and AI evaluations, suggesting that teachers applied their profession-specific expertise to the assessment process while AI produced somewhat generic evaluations. Overall, the study results indicate that AI can enhance the efficiency of formative assessments while providing timely feedback to learners.