Grading by AI Makes Me Feel Fairer? How Different Evaluators Affect College Students' Perception of Fairness

Fangyuan Chai^1* Jiajia Ma¹ Yi Wang¹ Jun Zhu^1* Tingting Han^2*

• ¹Beijing Foreign Studies University, China
• ²Hubei University of Economics, China

In the field of education, the use of new technologies has significantly enhanced the objectivity and scientificity of educational evaluation. However, concerns have been raised about the fairness of evaluators, such as artificial intelligence (AI) algorithms.This research aimed to assess college students' perceptions of fairness in educational evaluation scenarios through three studies using experimental vignettes. The studies involved 172 participants in Study 1, 149 in Study 2, and 145 in Study 3. Study 1 found that different evaluators could significantly influence the fairness perception under three evaluation contexts. Students perceived AI algorithms as fairer than teachers.Study 2 revealed that information transparency was a mediator, indicating that students perceived higher fairness with AI algorithms due to increased transparency compared with teachers. Study 3 revealed that the explanation of evaluation outcomes moderated the effect of evaluator on students' perceived fairness. Specifically, when provided with explanations for evaluation results, the effect of evaluator on students' perceived fairness was lessened. In conclusion, this study emphasizes the importance of information transparency and comprehensive explanations in the evaluation process, which is more crucial than solely focusing on the type of evaluators. It also draws attention to potential risks like algorithmic hegemony and advocates for ethical considerations, including privacy regulations, in integrating new technologies into educational evaluation systems. Overall, this study provides valuable theoretical insights and practical guidance for conducting fairer educational evaluations in the era.