Comparing Machine Learning and Artificial Neural Network Models in Psychological Research: A ROC-Based Analysis

Marie-Luise Leitner^*Martin Arendasy

• University of Graz, Graz, Austria

Growing interest in data-driven selection has led psychologists to explore whether artificial neural network models offer advantages over established machine-learning approaches in applied settings. The present study investigated whether traditional machine learning (ML) or artificial neural network (ANN) models provide superior predictive accuracy in applied psychological selection contexts. To date, few studies have directly compared these approaches using Receiver Operating Characteristic (ROC)-based evaluation in large, real-world admissions datasets. Three traditional algorithms—logistic regression, decision tree, and random forest—were compared with a feedforward neural network (one hidden layer, 32 units) for predicting admission success in a university entrance examination. The dataset comprised N = 4,155 applicants and included sociodemographic, academic, and cognitive indicators. All models were implemented in Python (Scikit-learn, TensorFlow, Keras) and evaluated using accuracy and Receiver Operating Characteristic (ROC) analyses. Logistic regression achieved the highest performance (accuracy = .973, AUC = .99), closely followed by random forest (.961, AUC = .98). The neural network reached .933 accuracy but exhibited indications of overfitting, limiting its generalizability. Feature importance analyses highlighted biology, chemistry, and numerical reasoning as the most predictive variables. Overall, the findings indicate that for medium-sized, structured psychological datasets, traditional machine-learning models remain more stable, interpretable, and robust than the evaluated shallow neural network architecture. These results underscore the importance of rigorous model selection and provide practical guidance for integrating predictive analytics into applied psychological assessment.