<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" article-type="research-article" dtd-version="2.3" xml:lang="EN">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Educ.</journal-id>
<journal-title>Frontiers in Education</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Educ.</abbrev-journal-title>
<issn pub-type="epub">2504-284X</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/feduc.2023.1106679</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Education</subject>
<subj-group>
<subject>Original Research</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Artificial neural network model to predict student performance using nonpersonal information</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<name>
<surname>Chavez</surname>
<given-names>Heyul</given-names>
</name>
<xref rid="aff1" ref-type="aff"><sup>1</sup></xref>
<xref rid="aff2" ref-type="aff"><sup>2</sup></xref>
<uri xlink:href="https://loop.frontiersin.org/people/2114799/overview"/>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Chavez-Arias</surname>
<given-names>Bill</given-names>
</name>
<xref rid="aff3" ref-type="aff"><sup>3</sup></xref>
<uri xlink:href="https://loop.frontiersin.org/people/2190661/overview"/>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Contreras-Rosas</surname>
<given-names>Sebastian</given-names>
</name>
<xref rid="aff3" ref-type="aff"><sup>3</sup></xref>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Alvarez-Rodr&#x00ED;guez</surname>
<given-names>Jose Mar&#x00ED;a</given-names>
</name>
<xref rid="aff4" ref-type="aff"><sup>4</sup></xref>
<uri xlink:href="https://loop.frontiersin.org/people/1978682/overview"/>
</contrib>
<contrib contrib-type="author" corresp="yes">
<name>
<surname>Raymundo</surname>
<given-names>Carlos</given-names>
</name>
<xref rid="aff5" ref-type="aff"><sup>5</sup></xref>
<xref rid="c001" ref-type="corresp"><sup>&#x002A;</sup></xref>
<uri xlink:href="https://loop.frontiersin.org/people/2112260/overview"/>
</contrib>
</contrib-group>
<aff id="aff1"><sup>1</sup><institution>Telecommunications and Networking Engineering, Universidad Peruana de Ciencias Aplicadas</institution>, <addr-line>Lima</addr-line>, <country>Peru</country></aff>
<aff id="aff2"><sup>2</sup><institution>The International Doctoral School, Universidad Rey Juan Carlos</institution>, <addr-line>Madrid</addr-line>, <country>Spain</country></aff>
<aff id="aff3"><sup>3</sup><institution>Computer Sciences School, Universidad Peruana de Ciencias Aplicadas</institution>, <addr-line>Lima</addr-line>, <country>Peru</country></aff>
<aff id="aff4"><sup>4</sup><institution>Department of Computer Science and Engineering, Universidad Carlos III</institution>, <addr-line>Madrid</addr-line>, <country>Spain</country></aff>
<aff id="aff5"><sup>5</sup><institution>Research Department, Universidad Peruana de Ciencias Aplicadas</institution>, <addr-line>Lima</addr-line>, <country>Peru</country></aff>
<author-notes>
<fn id="fn0001" fn-type="edited-by"><p>Edited by: Eduardo Cascallar, KU Leuven, Belgium</p></fn>
<fn id="fn0002" fn-type="edited-by"><p>Reviewed by: Tak-Lam Wong, Douglas College, Canada; Jacqueline Whalley, Auckland University of Technology, New Zealand</p></fn>
<corresp id="c001">&#x002A;Correspondence: Carlos Raymundo, <email>carlos.raymundo@upc.edu.pe</email></corresp>
<fn id="fn0003" fn-type="other"><p>This article was submitted to Assessment, Testing and Applied Measurement, a section of the journal Frontiers in Cardiovascular Medicine</p></fn>
</author-notes>
<pub-date pub-type="epub">
<day>09</day>
<month>02</month>
<year>2023</year>
</pub-date>
<pub-date pub-type="collection">
<year>2023</year>
</pub-date>
<volume>8</volume>
<elocation-id>1106679</elocation-id>
<history>
<date date-type="received">
<day>25</day>
<month>11</month>
<year>2022</year>
</date>
<date date-type="accepted">
<day>25</day>
<month>01</month>
<year>2023</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x00A9; 2023 Chavez, Chavez-Arias, Contreras-Rosas, Alvarez-Rodr&#x00ED;guez and Raymundo.</copyright-statement>
<copyright-year>2023</copyright-year>
<copyright-holder>Chavez, Chavez-Arias, Contreras-Rosas, Alvarez-Rodr&#x00ED;guez and Raymundo</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/"><p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p>
</license>
</permissions>
<abstract>
<p>In recent years, artificial intelligence has played an important role in education, wherein one of the most commonly used applications is forecasting students&#x2019; academic performance based on personal information such as social status, income, address, etc. This study proposes and develops an artificial neural network model capable of determining whether a student will pass a certain class without using personal or sensitive information that may compromise student privacy. For model training, we used information regarding 32,000 students collected from The Open University of the United Kingdom, such as number of times they took the course, average number of evaluations, course pass rate, average use of virtual materials per date and number of clicks in virtual classrooms. Attributes selected for the model are as follows: 93.81% accuracy, 94.15% precision, 95.13% recall, and 94.64% F1-score. These results will help the student authorities to take measures to avoid withdrawal and underachievement.</p>
</abstract>
<kwd-group>
<kwd>privacy</kwd>
<kwd>personal data</kwd>
<kwd>neural networks</kwd>
<kwd>forecasting</kwd>
<kwd>academic performance</kwd>
</kwd-group>
<counts>
<fig-count count="8"/>
<table-count count="3"/>
<equation-count count="7"/>
<ref-count count="29"/>
<page-count count="11"/>
<word-count count="7511"/>
</counts>
</article-meta>
</front>
<body>
<sec id="sec1" sec-type="intro">
<label>1.</label>
<title>Introduction</title>
<p>Artificial intelligence has exerted a large impact in most industries, including education. The use of these technologies (Deep Learning, Automatization and Natural Language Processing) provides important knowledge that benefits the industry, such as providing better understanding of the student learning processes or improving their results.</p>
<p>Low academic performance is one of the main problems facing higher education worldwide. One reason why this issue has become difficult to solve is that academic performance is influenced by several factors. <xref ref-type="bibr" rid="ref29">Tejedor and Garc&#x00ED;a-Valc&#x00E1;rcel (2007)</xref> mentions five factors: identification, psychological, academic, pedagogical, and socio-family. Similarly, there can be various consequences of poor academic performance, including academic attrition. According to <xref ref-type="bibr" rid="ref32">Vicerrectorado Acad&#x00E9;mico (2017)</xref>, one of the main factors for academic desertion is poor academic performance. Within the same context, <xref ref-type="bibr" rid="ref31">Viale (2014)</xref> argues that the percentage of students who fail classes in the first cycle is usually high, but when students repeat a class and fail it again, many decide to drop out of university. Furthermore, low academic performance is considered a quality indicator for educational institutions, so its assessment is critical for these institutions. In Peru, <xref ref-type="bibr" rid="ref27">SINEACE (2016)</xref> states that following up on students and leveling their deficiencies to prevent low academic performance is considered a quality standard for university programs and to achieve accreditation.</p>
<p>The COVID-19 pandemic resulted in students migrating to virtual learning environments. This introduced the need for students and teachers to adapt to a hybrid learning environment, if necessary, where they face different obstacles. <xref ref-type="bibr" rid="ref20">Liao and Wu (2022)</xref> mentioned that hybrid learning has provided students with more professional growth opportunities; however, this has also become a challenge because students are exposed to different types of distractions during the learning process.</p>
<p>In their research, <xref ref-type="bibr" rid="ref5">Blanco et al. (2016)</xref> and <xref ref-type="bibr" rid="ref24">Rincon-Flores et al. (2020)</xref> forecast academic student performance using different algorithms. <xref ref-type="bibr" rid="ref5">Blanco et al. (2016)</xref> proposed a deep neural network to predict academic performance, achieving 78% accuracy, whereas <xref ref-type="bibr" rid="ref24">Rincon-Flores et al. (2020)</xref> mentioned several models, such as K-nearest neighbors (KNN) and Random Forest, to achieve 80% accuracy. However, these works do not justify why they are forecasting academic student performance, the application of these predictions, and who they are going to help.</p>
<p>This research seeks to propose and develop a model for the prediction of student academic performance, due to the large amount of data to be used, it is intended through deep neural networks to respond to the educational problem of low academic performance in universities. Forecasting academic student performance allows institutions to better identify at-risk students and take supporting measures, thereby ensuring that students successfully complete their academic cycles, in turn, helping institutions gain more prestige.</p>
<p>The model training data used will be provided by <xref ref-type="bibr" rid="ref18">Kuzilek et al. (2017)</xref>, wherein we can acquire information such as courses, exams, personal student information, and student interaction with the resources of the Open University&#x2019;s online academic platform. First, we apply preprocessing techniques to prepare the data that will be used for training. Second, we sort the students into pass or fail categories. Third, we split the data into training and testing datasets. Finally, we train the model using the corresponding dataset. As part of our results, the accuracy of our prediction model will be compared with other similar works.</p>
<p>This research seeks to answer the following question: How to estimate whether a student will pass a subject before completing 50% of the course without using personal information?</p>
</sec>
<sec id="sec2">
<label>2.</label>
<title>State of the art</title>
<sec id="sec3">
<label>2.1.</label>
<title>Using the Na&#x00EF;ve Bayes algorithm for forecasting students&#x2019; academic performance</title>
<p>The literature offers various mathematical applications, wherein the Na&#x00EF;ve Bayes algorithm stands out due to its large application percentage. In this section, we will discuss the success percentages of each study that used the Na&#x00EF;ve Bayes algorithm as well as their characteristics, such as the quantity of information required and under what circumstances each study was conducted.</p>
<p>In their research, <xref ref-type="bibr" rid="ref17">Kumar et al. (2020)</xref> assessed student performance in technical exams used for university application, an important process for universities. Forecasting whether students would pass or fail a technical exam based on their performance in certain subjects of their course was the technique used to approach the problem. The authors used data regarding 200&#x2009;K&#x2009;L University students from 2013 to 2017. Students were sorted and classified using two different algorithms: K-means clustering and hierarchical clustering. Once classified, the authors used the Na&#x00EF;ve Bayes algorithm for their predictions at 72% accuracy.</p>
<p>In their study, <xref ref-type="bibr" rid="ref15">Jayaprakash et al. (2020)</xref> discuss student dropout rates in the early stages of the program by calculating the factors that exert the most influence on academic student performance. For these predictions, the authors used 887 instances and 19 variables. Additionally, they used the Na&#x00EF;ve Bayes algorithm at 85.7% accuracy.</p>
<p>In their work, <xref ref-type="bibr" rid="ref8">Daud et al. (2017)</xref> assessed the reasons why many students drop out of their careers. For this, the study predicted personal, family, and economic factors that most influence student performance to determine whether any given student would complete their studies. For testing, the dataset is cleaned to obtain 50 students who finished their studies and 50 who did not. As part of their results, the Na&#x00EF;ve Bayes algorithm achieved 84.8% in the F1-score.</p>
<p>The study reported by <xref ref-type="bibr" rid="ref33">Widyaningsih et al. (2019)</xref> intended to forecast academic performance for freshman students using a semisupervised learning approach to classify student performance. In this study, the Na&#x00EF;ve Bayes classifier was used as a methodology, which achieved 96% accuracy.</p>
<p><xref ref-type="bibr" rid="ref10">Devasia et al. (2016)</xref> addressed student desertion in higher education. The proposed system is a web-based application that employs the Na&#x00EF;ve Bayesian mining technique to extract useful information. According to this study, the basic algorithm provides more accuracy in this field than other methods such as regression, decision tree, and neural networks.</p>
<p><xref ref-type="bibr" rid="ref2">Amazona and Hernandez (2019)</xref> forecast student performance to improve decision-making education. Educational data mining was used to model student academic performance through the Na&#x00EF;ve Bayes algorithm, assess the dataset, and predict academic performance. Results reveal that even when the Na&#x00EF;ve Bayes classifier is outperformed by other algorithms, it achieved an overall prediction rate of 88% in classification accuracy.</p>
<p>In their research, <xref ref-type="bibr" rid="ref23">P&#x00E1;ez and Guzm&#x00E1;n (2018)</xref> solved problems affecting several teaching and learning processes. To this end, they designed and automated a predictive model of student academic performance. The study was based on several algorithms, wherein the Na&#x00EF;ve Bayes outperformed the others at a 73% prediction rate.</p>
<p>In some cases, clustering is required for further data analysis. The aforementioned studies are aware of this fact because K-means is used in <xref ref-type="bibr" rid="ref17">Kumar et al. (2020)</xref> and <xref ref-type="bibr" rid="ref33">Widyaningsih et al. (2019)</xref>, hierarchical in <xref ref-type="bibr" rid="ref17">Kumar et al. (2020)</xref>, <xref ref-type="bibr" rid="ref8">Daud et al. (2017)</xref>, and <xref ref-type="bibr" rid="ref33">Widyaningsih et al. (2019)</xref>, and custom clustering types in <xref ref-type="bibr" rid="ref23">P&#x00E1;ez and Guzm&#x00E1;n (2018)</xref> and others depending on the desired level of classification. Nevertheless, the most common and easiest to use is Na&#x00EF;ve Bayes, which was used in <xref ref-type="bibr" rid="ref15">Jayaprakash et al. (2020)</xref>, <xref ref-type="bibr" rid="ref33">Widyaningsih et al. (2019)</xref>, and <xref ref-type="bibr" rid="ref2">Amazona and Hernandez (2019)</xref>. The type of data available is critical because more specific data will better cover student learning, academic environment issues, and more. Thus, as mentioned in <xref ref-type="bibr" rid="ref33">Widyaningsih et al. (2019)</xref>, groups must be as differentiated as possible.</p>
</sec>
<sec id="sec4">
<label>2.2.</label>
<title>Using decision trees/random forests to predict student academic performance</title>
<p><xref ref-type="bibr" rid="ref17">Kumar et al. (2020)</xref> also proposed using the C5.0 algorithm, a decision tree algorithm, to predict student performance. For this, they used the exact same methodology as with Na&#x00EF;ve Bayes. As a result, they obtained 81% accuracy.</p>
<p>Similarly, <xref ref-type="bibr" rid="ref15">Jayaprakash et al. (2020)</xref> proposed the random forest algorithm and an in-house variation, the improved random forest algorithm. These algorithms yielded 91 and 93% accuracy, respectively, when predicting academic performance.</p>
<p>In their work, <xref ref-type="bibr" rid="ref8">Daud et al. (2017)</xref> used the C4.5 and CART algorithms to forecast whether students would complete their coursework. Through these algorithms, the authors achieved 76.6 and 71%, respectively, in F1-score.</p>
<p><xref ref-type="bibr" rid="ref22">Ma et al. (2018)</xref> proposed using deep neural networks to predict the passing rate of online students. They used data from Guo Pei Online Education. First, the features that most affect passing rate were filtered through the filter-type feature selection algorithm, which provided 27 representative features. Then, preprocessing and standardization were performed on the remaining dataset (27 features). The decision tree algorithm was used for prediction, and the grid search algorithm was used to optimize the decision tree. The following metrics were used with these algorithms: precision, recall, F1-score, and runtime. Before testing, the data were divided into pass and fail. Finally, using the grid search algorithm at 50% pass / 50% fail data, decision tree achieved 96% accuracy.</p>
<p>Of all the literature in this section, six authors used decision tree algorithms: <xref ref-type="bibr" rid="ref17">Kumar et al. (2020)</xref>, <xref ref-type="bibr" rid="ref6">Castrill&#x00F3;n et al. (2020)</xref>, <xref ref-type="bibr" rid="ref8">Daud et al. (2017)</xref>, <xref ref-type="bibr" rid="ref22">Ma et al. (2018)</xref>, <xref ref-type="bibr" rid="ref12">Hasan et al. (2019)</xref>, and <xref ref-type="bibr" rid="ref2">Amazona and Hernandez (2019)</xref>, while <xref ref-type="bibr" rid="ref15">Jayaprakash et al. (2020)</xref>, <xref ref-type="bibr" rid="ref24">Rincon-Flores et al. (2020)</xref>, and <xref ref-type="bibr" rid="ref3">Benablo et al. (2018)</xref> used random forest algorithms. Despite random forests being a subset of decision trees, most studies use decision trees for multiple-variable predictions, such as those for academic performance.</p>
<p>In their study, <xref ref-type="bibr" rid="ref12">Hasan et al. (2019)</xref> present a model that tries to predict final exam results for a given student. For this, they used a dataset of 1,170 students in 3 courses. Then, the authors preprocessed the dataset by removing unnecessary columns such as Student ID. They used the KNN algorithm and a decision tree classifier (the ID3 algorithm) for their predictions, thereby obtaining 94.44% accuracy based on the decision tree classifier algorithm.</p>
<p>In their research, <xref ref-type="bibr" rid="ref3">Benablo et al. (2018)</xref> forecast student performance based on social network data, such as Facebook, Twitter, Instagram, and YouTube, as well as online games. A single-format input was created to serve as the training dataset following the attribute relationship file format. To finalize the model, the researchers considered using support vector machine (SVM), KNN, and random forest for prediction. Furthermore, to ensure that the model accurately classified a given dataset, a 10-fold cross validation was performed using the WEKA software. For these modes, three metrics were used: precision, recall, and F1-score. For the results, the authors used three types of samples (30, 50, and 100 instances). After testing 100 instances, random forest achieved a result of 100% in precision, 80.6% in recall, and 89.3% in F1-score.</p>
<p><xref ref-type="bibr" rid="ref2">Amazona and Hernandez (2019)</xref> proposed to use a decision tree for prediction following the same methodology as <xref ref-type="bibr" rid="ref17">Kumar et al. (2020)</xref> and <xref ref-type="bibr" rid="ref22">Ma et al. (2018)</xref>. The results from the decision tree were 93% in precision, 96% in F1-score, and 100% in recall.</p>
<p>From <xref ref-type="bibr" rid="ref17">Kumar et al. (2020)</xref>, <xref ref-type="bibr" rid="ref6">Castrill&#x00F3;n et al. (2020)</xref>, <xref ref-type="bibr" rid="ref8">Daud et al. (2017)</xref>, <xref ref-type="bibr" rid="ref22">Ma et al. (2018)</xref>, <xref ref-type="bibr" rid="ref12">Hasan et al. (2019)</xref>, and <xref ref-type="bibr" rid="ref2">Amazona and Hernandez (2019)</xref>, <xref ref-type="bibr" rid="ref24">Rincon-Flores et al. (2020)</xref> reported the highest accuracy rate using a decision tree algorithm at 96%. Moreover, it considered the second-highest number of forecasting variables (27).</p>
</sec>
<sec id="sec5">
<label>2.3.</label>
<title>Using neural networks to predict student academic performance</title>
<p>The study by <xref ref-type="bibr" rid="ref5">Blanco et al. (2016)</xref> focuses on predicting student results for the Data Structures I and II classes because these computer engineering classes present a significant degree of difficulty and require serious dedication and rigor from students, which means that their grades are often lower than desirable. Therefore, the authors designed a model based on the MATLAB system that predicts academic results in these classes after having been trained with specific data from each class. The model&#x2019;s architecture is designed with three layers, wherein an activation function was used for each neuron: for the input and hidden layers, the sigmoidal hyperbolic tangent function was used. For the output layer, the linear function was used because the authors wanted to achieve the largest possible range amplitude in the output interval, which also facilitated result interpretation. In this study, a prediction effectiveness of over 78% was achieved for the first subject and 75% for the second subject.</p>
<p>The research reported by <xref ref-type="bibr" rid="ref26">Sekeroglu et al. (2019)</xref> was developed based on the need to improve AI-based systems in the field of education because in-class and remote students often need help to improve their performance. Therefore, two neural network models were used: backpropagation (BP) and long/short-term memory (LSTM). The former uses a gradient descent algorithm during learning and propagates the error to update weights and minimize error values; the latter memorizes previous neural network inputs to provide more accurate results. The results were as follows: BP (70% variance) and LSTM (77.9% variance).</p>
<p><xref ref-type="bibr" rid="ref2">Amazona and Hernandez (2019)</xref> also proposed using a deep learning neural network model for prediction. The same methodology as above was used. The deep learning results were as follows: 98% precision, 97% F1-score, and 98% recall.</p>
<p>Of the studies carried out by <xref ref-type="bibr" rid="ref5">Blanco et al. (2016)</xref>, <xref ref-type="bibr" rid="ref2">Amazona and Hernandez (2019)</xref>, and <xref ref-type="bibr" rid="ref26">Sekeroglu et al. (2019)</xref>, the neural network that reported the highest accuracy was the one reported by <xref ref-type="bibr" rid="ref2">Amazona and Hernandez (2019)</xref> at 98% precision, 98% recall and 97% F1-score. Moreover, <xref ref-type="bibr" rid="ref2">Amazona and Hernandez (2019)</xref> used the least number of input variables at only 9.</p>
<p>In this sense, <xref ref-type="bibr" rid="ref5">Blanco et al. (2016)</xref> used the highest number of input variables, 21. Nevertheless, its effectiveness rate when predicting student performance was considerably low (75% accuracy) compared with the effectiveness reported by <xref ref-type="bibr" rid="ref2">Amazona and Hernandez (2019)</xref> (95% accuracy). It may be observed that <xref ref-type="bibr" rid="ref22">Ma et al. (2018)</xref> uses fewer layers and, therefore, fewer network neurons, which leads to a more inaccurate process.</p>
</sec>
<sec id="sec6">
<label>2.4.</label>
<title>Support vector machine (SVM)/support vector regression (SVR) prediction models to forecast student academic performance</title>
<p><xref ref-type="bibr" rid="ref8">Daud et al. (2017)</xref>, used a supervised learning model to predict whether students will complete or abandon their study programs. Specifically, they used the SVM model, wherein the best result was obtained with 86% in the F1-score test.</p>
<p><xref ref-type="bibr" rid="ref22">Ma et al. (2018)</xref> also used the SVM supervised learning model to predict online student passing rates. Using the grid search algorithm at 50% pass / 50% fail data, this model achieved 95% accuracy.</p>
<p><xref ref-type="bibr" rid="ref26">Sekeroglu et al. (2019)</xref>, used the SVR model to improve AI-based systems in the field of education. This model provided predictions at 79.7% variance.</p>
<p><xref ref-type="bibr" rid="ref3">Benablo et al. (2018)</xref> forecasted student performance based on social network data, such as Facebook, Twitter, Instagram, and YouTube, using the SVM model. In a 100-instance test, this model reported 100% precision, 96.8% recall and 98.4% F1-score, thereby being the most effective model.</p>
<p>In their study, <xref ref-type="bibr" rid="ref21">Liao et al. (2019)</xref> sought to predict which students are at risk of achieving poor performance in a given class. This must be identified early enough to allow instructors to help students before they fall behind. Thus, they designed a model that predicts the student&#x2019;s final exam grade using a binary SVM classifier, which is trained with the radial basis function kernel based on the selected course. Additionally, the model is based on three parameters that indicate the severity of different classification error types. Based on this model, at least 62% of the at-risk students were correctly identified.</p>
<p>From <xref ref-type="bibr" rid="ref8">Daud et al. (2017)</xref>, <xref ref-type="bibr" rid="ref22">Ma et al. (2018)</xref>, <xref ref-type="bibr" rid="ref26">Sekeroglu et al. (2019)</xref>, <xref ref-type="bibr" rid="ref3">Benablo et al. (2018)</xref>, and <xref ref-type="bibr" rid="ref21">Liao et al. (2019)</xref>, the supervised learning model that reported the highest accuracy was the one used by <xref ref-type="bibr" rid="ref3">Benablo et al. (2018)</xref> at 100% precision, 96.8% recall, and 98.4% F1-score. To reach this accuracy, the authors used cross validation 10 times to avoid overfitting the model. Here, we can see a pattern: <xref ref-type="bibr" rid="ref26">Sekeroglu et al. (2019)</xref> did not use cross validation, and it reported the lowest accuracy rate at 79.7%. Similarly, <xref ref-type="bibr" rid="ref8">Daud et al. (2017)</xref> performed cross validation but only five times, which is half as many times as <xref ref-type="bibr" rid="ref3">Benablo et al. (2018)</xref>. <xref ref-type="bibr" rid="ref8">Daud et al. (2017)</xref> reported an accuracy rate of 86%, which is considerably high but not as high as the accuracy rates reported by <xref ref-type="bibr" rid="ref21">Liao et al. (2019)</xref>, <xref ref-type="bibr" rid="ref22">Ma et al. (2018)</xref>, and <xref ref-type="bibr" rid="ref3">Benablo et al. (2018)</xref>, who performed the cross validation 10 times.</p>
</sec>
<sec id="sec7">
<label>2.5.</label>
<title>Comparison of models</title>
<p>This section will summarize and compare the most relevant models in the literature, which will be shown in <xref rid="tab1" ref-type="table">Table 1</xref>. Data such as the technique used, accuracy, precision, recall, etc. were analyzed. It should be noted that not all models had all validation metrics.</p>
<table-wrap position="float" id="tab1">
<label>Table 1</label>
<caption>
<p>Model comparison.</p>
</caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th align="left" valign="top">Author</th>
<th align="left" valign="top">Technique</th>
<th align="left" valign="top">Dataset</th>
<th align="left" valign="top">Use of personal information</th>
<th align="center" valign="top">Accuracy</th>
<th align="center" valign="top">Precision</th>
<th align="center" valign="top">Recall</th>
<th align="center" valign="top">F1-score</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" valign="top">Proposed model</td>
<td align="left" valign="top">R.N.</td>
<td align="left" valign="top">OU</td>
<td align="left" valign="top">NO</td>
<td align="center" valign="top">93.81%</td>
<td align="center" valign="top">94.15%</td>
<td align="center" valign="top">95.13%</td>
<td align="center" valign="top">94.64%</td>
</tr>
<tr>
<td align="left" valign="top">
<xref ref-type="bibr" rid="ref3">Benablo et al. (2018)</xref>
</td>
<td align="left" valign="top">SVM</td>
<td align="left" valign="top">Private</td>
<td align="left" valign="top">YES</td>
<td align="center" valign="top">-</td>
<td align="center" valign="top">98.1%</td>
<td align="center" valign="top">100.0%</td>
<td align="center" valign="top">99.0%</td>
</tr>
<tr>
<td align="left" valign="top">
<xref ref-type="bibr" rid="ref3">Benablo et al. (2018)</xref>
</td>
<td align="left" valign="top">KNN</td>
<td align="left" valign="top">Private</td>
<td align="left" valign="top">YES</td>
<td align="center" valign="top">-</td>
<td align="center" valign="top">98.1%</td>
<td align="center" valign="top">99.0%</td>
<td align="center" valign="top">98.6%</td>
</tr>
<tr>
<td align="left" valign="top">
<xref ref-type="bibr" rid="ref25">Rodr&#x00ED;giez-Hern&#x00E1;ndez et al. (2021)</xref>
</td>
<td align="left" valign="top">Na&#x00EF;ve Bayes</td>
<td align="left" valign="top">ICFES (<xref ref-type="bibr" rid="ref14">Instituto Colombiano para la Evaluaci&#x00F3;n de la Educaci&#x00F3;n &#x2013; ICFES, 2021</xref>)</td>
<td align="left" valign="top">YES</td>
<td align="center" valign="top">96.0%</td>
<td align="center" valign="top">85.7%</td>
<td align="center" valign="top">89.0%</td>
<td align="center" valign="top">87.3%</td>
</tr>
<tr>
<td align="left" valign="top">
<xref ref-type="bibr" rid="ref2">Amazona and Hernandez (2019)</xref>
</td>
<td align="left" valign="top">Decision Tree</td>
<td align="left" valign="top">ASCOT (<xref ref-type="bibr" rid="ref9">Department of Information Technology. Aurora State College of Technology, n.d.</xref>)</td>
<td align="left" valign="top">YES</td>
<td align="center" valign="top">93.0%</td>
<td align="center" valign="top">93.0%</td>
<td align="center" valign="top">100.0%</td>
<td align="center" valign="top">93.0%</td>
</tr>
<tr>
<td align="left" valign="top">
<xref ref-type="bibr" rid="ref35">Ya&#x011F;c&#x0131; (2022)</xref>
</td>
<td align="left" valign="top">R.N.</td>
<td align="left" valign="top">Turkish&#x2019; SIS (<xref ref-type="bibr" rid="ref35">Ya&#x011F;c&#x0131;, 2022</xref>)</td>
<td align="left" valign="top">YES</td>
<td align="center" valign="top">86.3%</td>
<td align="center" valign="top">74.8%</td>
<td align="center" valign="top">74.6%</td>
<td align="center" valign="top">72.3%</td>
</tr>
<tr>
<td align="left" valign="top">
<xref ref-type="bibr" rid="ref36">Yousafzai et al. (2021)</xref>
</td>
<td align="left" valign="top">R.N.</td>
<td align="left" valign="top">Student Performance Data Set (<xref ref-type="bibr" rid="ref30">UCI Machine Learning Repository, 2021</xref>)</td>
<td align="left" valign="top">YES</td>
<td align="center" valign="top">90.1%</td>
<td align="center" valign="top">90.0%</td>
<td align="center" valign="top">90.0%</td>
<td align="center" valign="top">90.0%</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
<sec id="sec8">
<label>2.6.</label>
<title>Professional ethics</title>
<p>According to <xref ref-type="bibr" rid="ref1">Alaieri and Vellino (2017)</xref>, where different ethical regimes are applicable, artificial intelligence decisions must be reliable and duly justified so that users, manufacturers, and legislators can understand how decisions are made and what ethical principles were applied in each case. This study presents a model that breaks down the stages of ethical decision-making into their elementary components. The proposed project references ethical decision-making matters regarding the algorithm used to predict student performance. The ethical issues are detailed in Section 3.2.</p>
</sec>
</sec>
<sec id="sec9" sec-type="conclusions">
<label>3.</label>
<title>Contribution</title>
<p>This section introduces and explains the proposed neural network architecture and machine learning models to predict student academic performance.</p>
<sec id="sec10">
<label>3.1.</label>
<title>Architecture</title>
<p><xref rid="fig1" ref-type="fig">Figure 1</xref> illustrates the architecture used in the model proposed.</p>
<fig position="float" id="fig1">
<label>Figure 1</label>
<caption>
<p>Proposed architecture.</p>
</caption>
<graphic xlink:href="feduc-08-1106679-g001.tif"/>
</fig>
<sec id="sec11">
<label>3.1.1.</label>
<title>Objective definition</title>
<p>The main objective of this project is to forecast the academic performance of university students throughout their course to help professors and academic institutions better identify students at risk of failing and adopt corresponding supporting measures, thereby ensuring that students successfully complete their academic cycle, which in turn helps the institution garner prestige.</p>
</sec>
<sec id="sec12">
<label>3.1.2.</label>
<title>Information collection</title>
<p>The dataset used for project development was collected from the Open University Learning Analytics dataset, which provides information about courses, students, and their interactions with the Virtual Learning Environment (VLE). The dataset consists of tables interconnected with a single identifier. All tables are in .csv format. <xref rid="fig2" ref-type="fig">Figure 2</xref> denotes the dataset structure.</p>
<fig position="float" id="fig2">
<label>Figure 2</label>
<caption>
<p>Dataset structure <xref ref-type="bibr" rid="ref18">Kuzilek et al. (2017)</xref>.</p>
</caption>
<graphic xlink:href="feduc-08-1106679-g002.tif"/>
</fig>
<p>The dataset obtained from <xref ref-type="bibr" rid="ref18">Kuzilek et al. (2017)</xref> contains information about 22 courses, 32,593 students, their evaluation results, and records of their interactions with the VLE represented by daily summaries of student clicks (10,655,280 entries). This information is arranged in seven tables.</p>
</sec>
<sec id="sec13">
<label>3.1.3.</label>
<title>Information analysis</title>
<p>After the information has been collected, it must be analyzed. This analysis will help us segment the information and facilitate its manipulation. The dataset collected is detailed below, each set represents a table of the entire database.</p>
<list list-type="bullet">
<list-item>
<p>Courses.csv stores course information.</p>
</list-item>
<list-item>
<p>Assessments.csv stores information regarding course assessments.</p>
</list-item>
<list-item>
<p>Vle.csv contains information about the materials available on the VLE.</p>
</list-item>
<list-item>
<p>StudentInfo.csv stores general information about the students, such as demographics and final grade.</p>
</list-item>
<list-item>
<p>StudentRegistration.csv stores information about student registration to a course module.</p>
</list-item>
<list-item>
<p>StudentAssessment.csv contains information about student assessment results.</p>
</list-item>
<list-item>
<p>StudentVle.csv contains information about the student&#x2019;s interaction with the materials on the VLE.</p>
</list-item>
</list>
<p>The tables have a total of 41 attributes, of which 20 are identifiers (Primary Key and Foreign Key); of the remaining 21 attributes, 9 were discarded, since they did not have a direct relationship with the student&#x2019;s grade according to the documentation of the database itself.</p>
<p>Of the remaining 12 attributes, 3 contained personal information: age, gender and location, and another 3 attributes contained information not relevant to the application. This left only 6 attributes from the entire database that were useful for our purpose.</p>
<p>The target variable is &#x201C;final_result&#x201D; found in the &#x201C;studentInfo&#x201D; table as shown in <xref rid="fig2" ref-type="fig">Figure 2</xref>, this variable is a &#x201C;string,&#x201D; representing whether a student passed or failed the course, which was transformed to &#x201C;bulean&#x201D; so that it can then be entered into the model.</p>
</sec>
<sec id="sec14">
<label>3.1.4.</label>
<title>Filters and preprocessing</title>
<p>As can be seen in <xref rid="fig3" ref-type="fig">Figure 3</xref>, shows the correlation of the variables using the spearman method, and <xref rid="fig4" ref-type="fig">Figure 4</xref> shows the correlation of the variables but using the pearson method. After observing a correlation between the pass_rate and weighted_grade variables in the two figures, we applied a sklearn method (feature_importances) to see which variable had the greatest impact on both the machine learning models and the neural network.</p>
<fig position="float" id="fig3">
<label>Figure 3</label>
<caption>
<p>Spearman&#x2019;s correlation coefficient:</p>
</caption>
<graphic xlink:href="feduc-08-1106679-g003.tif"/>
</fig>
<fig position="float" id="fig4">
<label>Figure 4</label>
<caption>
<p>Pearson&#x2019;s correlation coefficient.</p>
</caption>
<graphic xlink:href="feduc-08-1106679-g004.tif"/>
</fig>
<p>Using the Feature_importances function, we observed that weighted_grade has more impact on the models results, so we tested all the models using pass_rate and removing it as input in the models, the results are shown in <xref rid="tab2" ref-type="table">Table 2</xref>.</p>
<table-wrap position="float" id="tab2">
<label>Table 2</label>
<caption>
<p>Accuracy comparison using the &#x201C;pass_rate&#x201D; variable.</p>
</caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th align="left" valign="top" rowspan="2">Models</th>
<th align="center" valign="top" colspan="2">Accuracy</th>
</tr>
<tr>
<th align="center" valign="top">With pass_rate</th>
<th align="center" valign="top">Without pass_rate</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" valign="top">ANN (Proposed model)</td>
<td align="center" valign="top">92.22%</td>
<td align="center" valign="top">93.81%</td>
</tr>
<tr>
<td align="left" valign="top">Random Forest</td>
<td align="center" valign="top">92.61%</td>
<td align="center" valign="top">92.81%</td>
</tr>
<tr>
<td align="left" valign="top">SVM</td>
<td align="center" valign="top">80.03%</td>
<td align="center" valign="top">80.03%</td>
</tr>
<tr>
<td align="left" valign="top">Na&#x00EF;ve Bayes</td>
<td align="center" valign="top">90.01%</td>
<td align="center" valign="top">92.21%</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>On Random Forest and SVM, the change is not as significant as on ANN and Naive Bayes, for these results the pass_rate column was removed as an input.</p>
<p>Once the dataset has been filtered and sorted, it is ready for input into the artificial intelligence engine.</p>
</sec>
<sec id="sec15">
<label>3.1.5.</label>
<title>Input</title>
<p>The inputs used to predict student academic performance in final exams are as follows:</p>
<list list-type="bullet">
<list-item>
<p>Number of attempts (num_of_prev_attempts): Number of times a student has taken or repeated the course.</p>
</list-item>
<list-item>
<p>Average grade of the evaluation (weighted_grade): Weighted grade obtained by the student in all the previous evaluations of the course. Not to be confused with the final exam.</p>
</list-item>
<list-item>
<p>Test score (exam_score): The student&#x2019;s score on this evaluation. The range is from 0 to 100. A score below 40 is interpreted as a Fail. Grades are in the range of 0 to 100.</p>
</list-item>
<list-item>
<p>Average date of use of materials (date): average of the dates on which the learner interacted with the materials in the EVE measured as the number of days since the presentation of the module.</p>
</list-item>
<list-item>
<p>Average number of clicks (sum_click): Average number of interactions the learner had with the course materials.</p>
</list-item>
</list>
<p>It should be noted that the inputs have a uniform distribution.</p>
</sec>
<sec id="sec16">
<label>3.1.6.</label>
<title>Output</title>
<p>The outputs will be used by professors to easily identify students with the highest risk of failing the course so they can decide, based on their expertise, what type of specific methodology can help these students pass the course. To predict academic performance, we defined two categories:</p>
<list list-type="bullet">
<list-item>
<p>Pass: The student will pass the course.</p>
</list-item>
<list-item>
<p>Fail: The student could fail the course.</p>
</list-item>
</list>
<p>The proposed artificial intelligence will use one of these categories to indicate the results.</p>
</sec>
<sec id="sec17">
<label>3.1.7.</label>
<title>Models and parameters</title>
<p>Three machine learning modes and one artificial neural network were proposed and developed; the architecture and characteristics of each model will be detailed below.</p>
<sec id="sec18">
<label>3.1.7.1.</label>
<title>Artificial neural network</title>
<p>The proposed neural network contains three layers (<xref rid="fig5" ref-type="fig">Figure 5</xref>):</p>
<list list-type="bullet">
<list-item>
<p>The input layer contains five neurons representing the number of variables in the clean dataset. Its activation function is &#x201C;ReLU,&#x201D; since the computational step and the gradient calculation, compared to other activation functions, is faster.</p>
</list-item>
<list-item>
<p>A hidden layer wherein the number of neurons will be selected using the following empirical method (<xref ref-type="bibr" rid="ref13">Heaton, 2009</xref>): &#x201C;The number of hidden neurons must be less than twice the size of the input layer.&#x201D; Therefore, this hidden layer will have eight neurons, with an activation function of &#x201C;ReLU.&#x201D;</p>
</list-item>
<list-item>
<p>The output layer is comprised of one neuron, which represents the result from the model prediction. Its activation function is &#x201C;sigmoid,&#x201D; this is because we have a boolean output.</p>
</list-item>
</list>
<fig position="float" id="fig5">
<label>Figure 5</label>
<caption>
<p>Neural network architecture chart.</p>
</caption>
<graphic xlink:href="feduc-08-1106679-g005.tif"/>
</fig>
<p>For training, the following hyper parameters were used:</p>
<list list-type="bullet">
<list-item>
<p>Epochs: 100</p>
</list-item>
<list-item>
<p>Activation Function: ReLU &#x0026; Sigmoid</p>
<list list-type="simple">
<list-item>
<p>&#x006F; ReLU:</p>
</list-item>
</list>
</list-item>
</list>
<disp-formula id="EQ1"><label>(1)</label><mml:math id="M1"><mml:mrow><mml:mi>max</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mn>0</mml:mn><mml:mi mathvariant="normal">,</mml:mi><mml:mi>x</mml:mi></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:math></disp-formula>
<list list-type="simple">
<list-item>
<p>&#x006F; Sigmoid:</p>
</list-item>
</list>
<disp-formula id="EQ2"><label>(2)</label><mml:math id="M2"><mml:mrow><mml:mi>S</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mrow><mml:mn>1</mml:mn><mml:mo>+</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mo>&#x2212;</mml:mo><mml:mi>x</mml:mi></mml:mrow></mml:msup></mml:mrow></mml:mfrac></mml:mrow></mml:math></disp-formula>
<list list-type="bullet">
<list-item>
<p>Optimizer: Adam</p>
</list-item>
</list>
<disp-formula id="EQ3"><label>(3)</label><mml:math id="M3"><mml:mrow><mml:msub><mml:mi>m</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mi>&#x03B2;</mml:mi><mml:msub><mml:mi>m</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>+</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>&#x03B2;</mml:mi></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mrow><mml:mo>[</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mo>&#x2202;</mml:mo><mml:mi>L</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x2202;</mml:mo><mml:msub><mml:mi>w</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:mrow></mml:mfrac></mml:mrow><mml:mo>]</mml:mo></mml:mrow></mml:mrow></mml:math></disp-formula>
<list list-type="bullet">
<list-item>
<p>Loss Function: Binary_CrossEntropy</p>
</list-item>
</list>
<disp-formula id="EQ4"><label>(4)</label><mml:math id="M4"><mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>p</mml:mi></mml:msub><mml:mrow><mml:mo>(</mml:mo><mml:mi>q</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mo>&#x2212;</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mi>N</mml:mi></mml:mfrac><mml:munderover><mml:mstyle displaystyle="true"><mml:mo>&#x2211;</mml:mo></mml:mstyle><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:munderover><mml:msub><mml:mi>y</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mspace width="thickmathspace"/><mml:mi>log</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mi>p</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:msub><mml:mi>y</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>+</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:msub><mml:mi>y</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>+</mml:mo><mml:mi>log</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>p</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:msub><mml:mi>y</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:math></disp-formula>
<list list-type="bullet">
<list-item>
<p>Metrics: Accuracy and Loss</p>
</list-item>
</list>
<disp-formula id="EQ5"><label>(5)</label><mml:math id="M5"><mml:mrow><mml:mi>A</mml:mi><mml:mi>c</mml:mi><mml:mi>c</mml:mi><mml:mi>u</mml:mi><mml:mi>r</mml:mi><mml:mi>a</mml:mi><mml:mi>c</mml:mi><mml:mi>y</mml:mi><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mi>N</mml:mi><mml:mi>u</mml:mi><mml:mi>m</mml:mi><mml:mi>b</mml:mi><mml:mi>e</mml:mi><mml:mi>r</mml:mi><mml:mi mathvariant="normal"></mml:mi><mml:mi>o</mml:mi><mml:mi>f</mml:mi><mml:mi mathvariant="normal"></mml:mi><mml:mi>c</mml:mi><mml:mi>o</mml:mi><mml:mi>r</mml:mi><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mi>c</mml:mi><mml:mi>t</mml:mi><mml:mi mathvariant="normal"></mml:mi><mml:mi>p</mml:mi><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>c</mml:mi><mml:mi>t</mml:mi><mml:mi>i</mml:mi><mml:mi>o</mml:mi><mml:mi>n</mml:mi><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mi>T</mml:mi><mml:mi>o</mml:mi><mml:mi>t</mml:mi><mml:mi>a</mml:mi><mml:mi>l</mml:mi><mml:mi mathvariant="normal"></mml:mi><mml:mi>n</mml:mi><mml:mi>u</mml:mi><mml:mi>m</mml:mi><mml:mi>b</mml:mi><mml:mi>e</mml:mi><mml:mi>r</mml:mi><mml:mi mathvariant="normal"></mml:mi><mml:mi>o</mml:mi><mml:mi>f</mml:mi><mml:mi mathvariant="normal"></mml:mi><mml:mi>p</mml:mi><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>c</mml:mi><mml:mi>t</mml:mi><mml:mi>i</mml:mi><mml:mi>o</mml:mi><mml:mi>n</mml:mi><mml:mi>s</mml:mi></mml:mrow></mml:mfrac><mml:mspace width="thickmathspace"/></mml:mrow></mml:math></disp-formula>
<disp-formula id="EQ6"><label>(6)</label><mml:math id="M6"><mml:mrow><mml:mi>log</mml:mi><mml:mspace width="thickmathspace"/><mml:mi>l</mml:mi><mml:mi>o</mml:mi><mml:mi>s</mml:mi><mml:mi>s</mml:mi><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mi>N</mml:mi></mml:mfrac><mml:munderover><mml:mstyle displaystyle="true"><mml:mo>&#x2211;</mml:mo></mml:mstyle><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:munderover><mml:munderover><mml:mstyle displaystyle="true"><mml:mo>&#x2211;</mml:mo></mml:mstyle><mml:mrow><mml:mi>j</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>M</mml:mi></mml:munderover><mml:msub><mml:mi>y</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mi>log</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:msub><mml:mi>p</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mspace width="thickmathspace"/></mml:mrow></mml:math></disp-formula>
<list list-type="bullet">
<list-item>
<p>Overfitting: EarlyStopping</p>
</list-item>
</list>
<disp-formula id="EQ7"><label>(7)</label><mml:math id="M7"><mml:mrow><mml:munder><mml:mstyle displaystyle="true"><mml:mo>&#x222B;</mml:mo></mml:mstyle><mml:mi>p</mml:mi></mml:munder><mml:mrow><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:munder><mml:mstyle displaystyle="true"><mml:mo>&#x222B;</mml:mo></mml:mstyle><mml:mi>Y</mml:mi></mml:munder><mml:mi>y</mml:mi><mml:mspace width="thickmathspace"/><mml:mi>d</mml:mi><mml:mi>p</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mi>y</mml:mi><mml:mi mathvariant="normal">|</mml:mi><mml:mi>x</mml:mi></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>,</mml:mo><mml:mi>x</mml:mi><mml:mspace width="thickmathspace"/><mml:mi>&#x03F5;</mml:mi><mml:mspace width="thickmathspace"/><mml:mi>X</mml:mi></mml:mrow></mml:math></disp-formula>
<sec id="sec19">
<label>3.1.7.1.1.</label>
<title>Training</title>
<p>Because overfitting is a common problem in neural networks, we implement the following strategy to prevent this issue. We divide the dataset 70/30 such that 70% is directed toward the training of the neural network and the remaining 30% is used for neural network validation.</p>
<p>The number of training epochs will be 100, and in each epoch, the backpropagation algorithm will be used to improve weights and input biases and obtain a more accurate result or &#x201C;minimize loss.&#x201D;</p>
</sec>
</sec>
<sec id="sec20">
<label>3.1.7.2.</label>
<title>SVM</title>
<p>To know exactly which parameters are necessary for the good performance of our model, we apply the sklearn function GrigSearchCV, which receives a set of parameters and returns the best ones for the model according to the training data, it is worth noting that this function was used for the following machine learning models.</p>
<p>As input for the GrigSearchCV function, we gave it the following parameters:</p>
<list list-type="bullet">
<list-item>
<p>C&#x2009;=&#x2009;[0.1, 1, 10, 100]</p>
</list-item>
<list-item>
<p>Gamma&#x2009;=&#x2009;[1, 0.1, 0.01, 0001]</p>
</list-item>
<list-item>
<p>Kernel&#x2009;=&#x2009;[&#x2018;rbf &#x2018;, &#x2018;poly&#x2019;, &#x2018;sigmoid&#x2019;]</p>
</list-item>
</list>
<p>The parameters used for the SVM model were as follows:</p>
<list list-type="bullet">
<list-item>
<p>C&#x2009;=&#x2009;100</p>
</list-item>
<list-item>
<p>Kernel&#x2009;=&#x2009;rbf</p>
</list-item>
<list-item>
<p>Gamma&#x2009;=&#x2009;0.1</p>
</list-item>
<list-item>
<p>Probability&#x2009;=&#x2009;True</p>
</list-item>
</list>
<p>The percentage for training was 70% of the dataset.</p>
</sec>
<sec id="sec21">
<label>3.1.7.3.</label>
<title>Na&#x00EF;ve Bayes</title>
<p>In the case of Naive Baye no parameters were specified, due to the same architecture, only the percentage for training had to be defined, which was the same as in the case of SVM, 70% of the dataset.</p>
</sec>
<sec id="sec22">
<label>3.1.7.4.</label>
<title>Random Forest</title>
<p>In the case of random fores, we used the GridSearchCV function mentioned above, with the following input parameters:</p>
<list list-type="bullet">
<list-item>
<p>n_estimators&#x2009;=&#x2009;[200, 700]</p>
</list-item>
<list-item>
<p>max_features&#x2009;=&#x2009;[&#x2018;auto&#x2019;, &#x2018;sqrt&#x2019;, &#x2018;log2&#x2019;]</p>
</list-item>
<list-item>
<p>max_depth&#x2009;=&#x2009;[2, 3, 4, 5, 6, 7, 8, 9, 10]</p>
</list-item>
</list>
<p>The parameters used for the Random Forest model were as follows:</p>
<list list-type="bullet">
<list-item>
<p>n_estimators&#x2009;=&#x2009;700</p>
</list-item>
<list-item>
<p>max_features&#x2009;=&#x2009;log2</p>
</list-item>
<list-item>
<p>max_depth&#x2009;=&#x2009;5</p>
</list-item>
</list>
<p>It is worth mentioning that in this model we used the feature_importances function which allows us to see which features are more relevant for the model according to the dataset, as a result we obtained weighted_grade as the most important feature, this can be seen in <xref rid="fig6" ref-type="fig">Figure 6</xref>.</p>
<fig position="float" id="fig6">
<label>Figure 6</label>
<caption>
<p>Important features.</p>
</caption>
<graphic xlink:href="feduc-08-1106679-g006.tif"/>
</fig>
</sec>
</sec>
</sec>
<sec id="sec23">
<label>3.2.</label>
<title>Ethics statement</title>
<sec id="sec24">
<label>3.2.1.</label>
<title>Discrimination criteria</title>
<p>Given the dataset variables used in the project, we handle some student personal information, such as gender. We use this variable neither for training the neural network nor for predictions. We believe that using this information will be unethical because it can generate discrimination regarding which gender is more &#x201C;intelligent&#x201D; or has greater capacities. We do not consider this relevant because a small section of the population does not appropriately represent all women or men worldwide. Furthermore, we do not want to create controversy in the classroom by implying that one group has better abilities than another; we are only trying to predict student performance in the course and not the abilities that a person may have.</p>
</sec>
<sec id="sec25">
<label>3.2.2.</label>
<title>Data exclusion</title>
<p>In Section 3.1.4, we mentioned some filters that will be applied to the dataset. One of these filters removes data from students who did not take some of the course assessments because they may be absent due to several reasons, such as illnesses, economic issues, and personal issues, which may have influenced the student&#x2019;s performance in the course. We believe that it would be unethical to deem a student inefficient when they may be experiencing circumstances that may prevent them from attending class and affect their academic performance.</p>
</sec>
<sec id="sec26">
<label>3.2.3.</label>
<title>Data manipulation for unethical purposes</title>
<p>The ethics model used in this project secures information provided by users because this information shall not be retained or used for unethical purposes. Given the number of cases where information is leaked with malicious intent, personal data was not required for the development of the project and was not used at any stage of the project.</p>
</sec>
</sec>
</sec>
<sec id="sec27" sec-type="results">
<label>4.</label>
<title>Results</title>
<p>This section presents the results from the proposed process as explained in the Contribution section. We must remember that the proposed algorithm is an artificial neural network, which predicts student academic performance in a given course.</p>
<p>The confusion matrix (<xref rid="fig7" ref-type="fig">Figure 7</xref>) of the model after training is shown below, where it is observed that out of a total of 291 passing students, 274 were predicted correctly, and out of a total of 210 failing students, 196 were predicted correctly.</p>
<fig position="float" id="fig7">
<label>Figure 7</label>
<caption>
<p>Confusion matrix.</p>
</caption>
<graphic xlink:href="feduc-08-1106679-g007.tif"/>
</fig>
<p><xref rid="tab3" ref-type="table">Table 3</xref> shows the results of the confusion matrices and the accuracy of the proposed and developed models, the compared models were trained and validated with the same data set.</p>
<table-wrap position="float" id="tab3">
<label>Table 3</label>
<caption>
<p>Results of the confusion matrices.</p>
</caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th align="left" valign="top">Models</th>
<th align="center" valign="top">True positives</th>
<th align="center" valign="top">False positive</th>
<th align="center" valign="top">True negatives</th>
<th align="center" valign="top">False negatives</th>
<th align="center" valign="top">Accuracy</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" valign="top">ANN (Proposed model)</td>
<td align="center" valign="top">274</td>
<td align="center" valign="top">17</td>
<td align="center" valign="top">196</td>
<td align="center" valign="top">14</td>
<td align="center" valign="top">93.81%</td>
</tr>
<tr>
<td align="left" valign="top">Random Forest</td>
<td align="center" valign="top">288</td>
<td align="center" valign="top">21</td>
<td align="center" valign="top">177</td>
<td align="center" valign="top">15</td>
<td align="center" valign="top">92.81%</td>
</tr>
<tr>
<td align="left" valign="top">SVM</td>
<td align="center" valign="top">294</td>
<td align="center" valign="top">15</td>
<td align="center" valign="top">107</td>
<td align="center" valign="top">85</td>
<td align="center" valign="top">80.03%</td>
</tr>
<tr>
<td align="left" valign="top">Na&#x00EF;ve Bayes</td>
<td align="center" valign="top">284</td>
<td align="center" valign="top">25</td>
<td align="center" valign="top">178</td>
<td align="center" valign="top">14</td>
<td align="center" valign="top">92.21%</td>
</tr>
</tbody>
</table>
</table-wrap>
<sec id="sec28">
<label>4.1.</label>
<title>Result interpretation</title>
<p>To track the evaluation of the training and validation process of the artificial neural network, the Accuracy and Loss metrics were used in each epoch.</p>
<p><xref rid="fig8" ref-type="fig">Figure 8</xref> illustrates the Accuracy and Loss recorded for each epoch. On one hand, Training Accuracy, in orange, reaches a certain percentage where it remains stable, and Validation Accuracy, in red, follows the same behavior as Validation Accuracy. On the other hand, Loss Training and Validation follow a similar decreasing pattern until reaching a minimum percentage, where they stabilize.</p>
<fig position="float" id="fig8">
<label>Figure 8</label>
<caption>
<p>Accuracy and loss recorded for each epoch.</p>
</caption>
<graphic xlink:href="feduc-08-1106679-g008.tif"/>
</fig>
<p>A total of 100 epochs were required for training in the hyperparameters. However, to avoid overfitting, EarlyStopping was used, thereby cutting off at 54 epochs upon detecting that the accuracy percentage remained constant.</p>
<p>The model achieved 93.81% accuracy, which indicates that the prediction was correct for the percentage of cases validated. Likewise, 94.15% precision was obtained. This measures the quality of the model, indicates the number of students who actually passed the course, and predicts that they would pass the course. In other words, it is the percentage of students correctly identified by the model as passing from the total number of students identified by the model as passing. The model achieved 95.13% recall, which indicates the number of approved students that the model identified, that is, the percentage of students correctly identified as approved from the total number of approved students. Finally, the model obtained an 94.64% F1-score, which is the combination of the accuracy and recall metrics as a single value.</p>
</sec>
<sec id="sec29">
<label>4.2.</label>
<title>Discussion</title>
<p>This section compares the performance of the proposed model with the models that had the best performance according to the literature. The most relevant studies are specified in <xref rid="tab1" ref-type="table">Table 1</xref>. Three machine learning models were implemented, which were trained with the same set of datasets with which the neural network was trained in order to compare them.</p>
<p>According to <xref ref-type="bibr" rid="ref3">Benablo et al. (2018)</xref>, who evaluated the SVM and KNN techniques, the SVM model provides 98% accuracy due to the type of model used to predict performance as a function of the input data. <xref ref-type="bibr" rid="ref3">Benablo et al. (2018)</xref> also details three other models used in which a variation in the instances of the dataset is seen. The higher the number of instances, the higher the accuracy, sometimes reaching 100%. After the implementation of our own SVM model, and the comparison with the proposed neural network model, we can observe which neural network is better both in accuracy and in the test results shown in the <xref rid="tab3" ref-type="table">Table 3</xref>.</p>
<p>After comparing with the <xref ref-type="bibr" rid="ref2">Amazona and Hernandez (2019)</xref> proposal, three different models were used: Na&#x00EF;ve Bayes, deep learning and decision tree, with an accuracy of 96, 98 and 93%, respectively. We opted to implement our own Na&#x00EF;ve Bayes model to train it on our dataset for comparative purposes. The results of the Na&#x00EF;ve Bayes model were the lowest in the comparisons performed, as can be seen in <xref rid="tab3" ref-type="table">Table 3</xref>.</p>
<p>In his research, <xref ref-type="bibr" rid="ref25">Rodr&#x00ED;giez-Hern&#x00E1;ndez et al. (2021)</xref> implemented an artificial neural network to predict the academic performance of senior students. The research notes that they used information from a sample of 162,030 students and that the trained model provided an accuracy of 82%. However, they also used personal information about the students (e.g., their socioeconomic status, household characteristics, personal background, the types of schools they had attended, and the salary of working students) to train the model. In this case, to perform the comparison of the models with the same data set, the <xref ref-type="bibr" rid="ref25">Rodr&#x00ED;giez-Hern&#x00E1;ndez et al. (2021)</xref> model would have to be modified, so a correct comparison cannot be performed; however, we can observe that 8 of the 10 attributes used as input data, are personal data, this means that if we remove the personal data from their model, the model would suffer a decay in performance. <xref ref-type="bibr" rid="ref35">Ya&#x011F;c&#x0131; (2022)</xref> implemented several models to predict students&#x2019; academic performance.</p>
<p>The models used were random forest, neural network, SVM, logistic regression, Na&#x00EF;ve Bayes and KNN, where the neural network provided the best results with an accuracy of 86.30%. <xref ref-type="bibr" rid="ref35">Ya&#x011F;c&#x0131; (2022)</xref> did not use students&#x2019; personal data. They only used their midterm and final exam grades, career path and academic department as information. The 86% accuracy obtained by this research study is the highest accuracy rate. Thus, a proprietary Random Forest model was implemented, trained with the same data set with which our proposed model was trained. The result of the Random Forest model was the best performing among the implemented models, this can be seen in <xref rid="tab3" ref-type="table">Table 3</xref>.</p>
<p>The research of <xref ref-type="bibr" rid="ref36">Yousafzai et al. (2021)</xref> compares the existing models, such as RNN, CNN, LSTM, SVM and BiLSTM, with the model proposed by the authors, which achieved a better accuracy of 90.16%. <xref ref-type="bibr" rid="ref36">Yousafzai et al. (2021)</xref> uses personal data, such as age, gender, address, parents&#x2019; occupation, type of school and parents&#x2019; educational level, to train the model. The dataset used for training is mainly composed of personal data. Similar case to <xref ref-type="bibr" rid="ref25">Rodr&#x00ED;giez-Hern&#x00E1;ndez et al. (2021)</xref>, due to the architecture developed by the authors and the database used, it is not possible to perform an equivalent comparison; however, we can observe that from the model proposed by <xref ref-type="bibr" rid="ref36">Yousafzai et al. (2021)</xref>, 15 of the 33 entries for his model are personal information of the students, such as: age, location, love relationship, father&#x2019;s jobs, mother&#x2019;s jobs, etc. which means that if we were to remove those 15 entries, the model would suffer a loss of performance.</p>
<p>Comparing our model with the literature, we can observe that personal data significantly affect the model results, regardless of the type of model used. For this reason, and due to the increasing vulnerability of personal data evidenced by data breaches and leaks, we propose a student performance prediction tool that does not expose sensitive student and teacher information and yields better results than the observed literature.</p>
</sec>
</sec>
<sec id="sec30" sec-type="conclusions">
<label>5.</label>
<title>Conclusion</title>
<p>In this research, an artificial neural network model was proposed to predict students&#x2019; academic performance in their courses, which obtained an accuracy of 93.81%, without exposing their personal data.</p>
<p>Low student performance such as student dropout in courses is a problem, which affects both educational institutions and students. Because of this, we developed a model superior to all the proposed literature that can predict whether a student will pass or fail a subject in the middle of the course. This information will be useful for the competent authorities of student achievement to take measures to avoid withdrawal and underachievement.</p>
<p>It was necessary to propose some ethical metrics for the development of our model. Unlike the literature, we proposed a neural network model that did not use personal input data and preserved students&#x2019; privacy. This gives rise to a new area of study in which personal data is avoided, as many industries today constantly report personal data breach and leakage.</p>
</sec>
<sec id="sec31">
<label>6.</label>
<title>Future work</title>
<p>In future work, more data regarding student interaction with digital platforms can be used, since as a consequence of the COVID-19 pandemic, academic learning has become largely virtualized and much more information will be available.</p>
<p>It is worth noting that this process of predicting student performance under ethical standards can be automated on university platforms, thus facilitating access to information for student affairs officers.</p>
</sec>
<sec id="sec32" sec-type="data-availability">
<title>Data availability statement</title>
<p>Publicly available datasets were analyzed in this study. This data can be found at: <ext-link xlink:href="https://analyse.kmi.open.ac.uk/open_dataset" ext-link-type="uri">https://analyse.kmi.open.ac.uk/open_dataset</ext-link>.</p>
</sec>
<sec id="sec33">
<title>Author contributions</title>
<p>HC, BC-A, and SC-R performed the predictive analysis. CR, HC, and BC-A wrote the first draft of the manuscript. CR, HC, BC-A, SC-R, and JA-R wrote sections of the manuscript. All authors contributed to the manuscript revision, read, and approved the submitted version.</p>
</sec>
<sec id="conf1" sec-type="COI-statement">
<title>Conflict of interest</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
<sec id="sec100" sec-type="disclaimer">
<title>Publisher&#x2019;s note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p>
</sec>
</body>
<back>
<ack>
<p>The authors would like to thank the Research Directorate of the Universidad Peruana de Ciencias Aplicadas for the support provided to carry out this research work through the UPC-EXPOST-2023-1 incentive.</p>
</ack>
<ref-list>
<title>References</title>
<ref id="ref1"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Alaieri</surname> <given-names>F.</given-names></name> <name><surname>Vellino</surname> <given-names>A.</given-names></name></person-group> (<year>2017</year>). &#x201C;<article-title>A decision making model for ethical (Ro)bots</article-title>.&#x201D; in <source>IEE International Symposium on Robotics and Intelligent Sensors (IRIS)</source>, <publisher-loc>Ottawa, ON, Canada</publisher-loc>. <fpage>203</fpage>&#x2013;<lpage>207</lpage>. doi: <pub-id pub-id-type="doi">10.1109/IRIS.2017.8250122</pub-id></citation></ref>
<ref id="ref2"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Amazona</surname> <given-names>M. V.</given-names></name> <name><surname>Hernandez</surname> <given-names>A. A.</given-names></name></person-group> (<year>2019</year>). &#x201C;<article-title>Modelling student performance using data mining techniques</article-title>.&#x201D; in <source>Proceedings of the 2019 5th International Conference on Computing and Data Engineering&#x2014;ICCDE&#x2019; 19</source>. <publisher-loc>New York, NY, USA</publisher-loc>: <publisher-name>Association for Computing Machinery</publisher-name>. <fpage>36</fpage>&#x2013;<lpage>40</lpage>. doi: <pub-id pub-id-type="doi">10.1145/3330530.3330544</pub-id></citation></ref>
<ref id="ref3"><citation citation-type="other"><person-group person-group-type="author"><name><surname>Benablo</surname> <given-names>C. I. P.</given-names></name> <name><surname>Sarte</surname> <given-names>E. T.</given-names></name> <name><surname>Dormido</surname> <given-names>J. M. D.</given-names></name> <name><surname>Palaoag</surname> <given-names>T.</given-names></name></person-group> (<year>2018</year>). <article-title>Higher education Student&#x2019;s academic performance analysis through predictive analytics</article-title> in <italic>Proceedings of the 2018 7th International Conference on Software and Computer Applications&#x2014;ICSCA 2018</italic>. New York, NY, USA: Association for Computing Machinery. <fpage>238</fpage>&#x2013;<lpage>242</lpage>. doi: <pub-id pub-id-type="doi">10.1145/3185089.3185102</pub-id></citation></ref>
<ref id="ref5"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Blanco</surname> <given-names>J.</given-names></name> <name><surname>Lovelle</surname> <given-names>S.</given-names></name> <name><surname>Fernandez</surname> <given-names>R.</given-names></name> <name><surname>Perez</surname> <given-names>E.</given-names></name></person-group> (<year>2016</year>). <article-title>Predicci&#x00F3;n de resultados acad&#x00E9;micos de estudiantes de inform&#x00E1;tica mediante el uso de redes neuronales</article-title>. <source>Ingeniare. Revista chilena de ingenier&#x00ED;a</source> <volume>24</volume>, <fpage>715</fpage>&#x2013;<lpage>727</lpage>. doi: <pub-id pub-id-type="doi">10.4067/S0718-33052016000400015</pub-id></citation></ref>
<ref id="ref6"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Castrill&#x00F3;n</surname> <given-names>O.</given-names></name> <name><surname>Sarache</surname> <given-names>W.</given-names></name> <name><surname>Ruiz</surname> <given-names>S.</given-names></name></person-group> (<year>2020</year>). <article-title>Prediction of academic performance using artificial intelligence techniques</article-title>. <source>From Univ.</source> <volume>13</volume>, <fpage>93</fpage>&#x2013;<lpage>102</lpage>. doi: <pub-id pub-id-type="doi">10.4067/S0718-50062020000100093</pub-id></citation></ref>
<ref id="ref8"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Daud</surname> <given-names>A.</given-names></name> <name><surname>Aljohani</surname> <given-names>N. R.</given-names></name> <name><surname>Abbasi</surname> <given-names>R. A.</given-names></name> <name><surname>Lytras</surname> <given-names>M. D.</given-names></name> <name><surname>Abbas</surname> <given-names>F.</given-names></name> <name><surname>Alowibdi</surname> <given-names>J. S.</given-names></name></person-group> (<year>2017</year>). &#x201C;<article-title>Predicting student performance using advanced learning analytics</article-title>.&#x201D; in <italic>Proceedings of the 26th International Conference on World Wide Web Companion&#x2014;WWW &#x2018;17 Companion</italic>. <publisher-loc>Republic and Canton of Geneva, CHE</publisher-loc>: <publisher-name>International World Wide Web Conferences Steering Committee</publisher-name>. <fpage>415</fpage>&#x2013;<lpage>421</lpage>. doi: <pub-id pub-id-type="doi">10.1145/3041021.3054164</pub-id></citation></ref>
<ref id="ref9"><citation citation-type="other"><person-group person-group-type="author"><collab id="coll1">Department of Information Technology. Aurora State College of Technology</collab></person-group>. Available at: <ext-link xlink:href="https://www.ascot.edu.ph/departments/department-of-information-technology/" ext-link-type="uri">https://www.ascot.edu.ph/departments/department-of-information-technology/</ext-link></citation></ref>
<ref id="ref10"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Devasia</surname> <given-names>T.</given-names></name> <name><surname>Vinushree</surname> <given-names>T. P.</given-names></name> <name><surname>Hegde</surname> <given-names>V.</given-names></name></person-group> (<year>2016</year>). &#x201C;<article-title>Prediction of students' performance using educational data mining</article-title>.&#x201D; in <source>2016 International Conference on Data Mining and Advanced Computing (SAPIENCE)</source>. <publisher-loc>Ernakulam, India</publisher-loc>. <fpage>91</fpage>&#x2013;<lpage>95</lpage>. doi: <pub-id pub-id-type="doi">10.1109/SAPIENCE.2016.7684167</pub-id></citation></ref>
<ref id="ref12"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Hasan</surname> <given-names>H. M. R.</given-names></name> <name><surname>Rabby</surname> <given-names>A. S. A.</given-names></name> <name><surname>Islam</surname> <given-names>M. T.</given-names></name> <name><surname>Hossain</surname> <given-names>S. A.</given-names></name></person-group> (<year>2019</year>). &#x201C;<article-title>Machine learning algorithm for Student&#x2019;s performance prediction</article-title>.&#x201D; <source>2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT)</source>. <publisher-loc>Kanpur, India</publisher-loc>. <fpage>1</fpage>&#x2013;<lpage>7</lpage>. doi: <pub-id pub-id-type="doi">10.1109/ICCCNT45670.2019.8944629</pub-id></citation></ref>
<ref id="ref13"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Heaton</surname> <given-names>J.</given-names></name></person-group> (<year>2009</year>). <source>Introduction to Neural Networks with Java</source>, <edition>2nd</edition>. <publisher-loc>Chesterfield, MO</publisher-loc>: <publisher-name>Heaton Research</publisher-name>, 2009, p. <fpage>129</fpage>.</citation></ref>
<ref id="ref14"><citation citation-type="other"><person-group person-group-type="author"><collab id="coll2">Instituto Colombiano para la Evaluaci&#x00F3;n de la Educaci&#x00F3;n &#x2013; ICFES</collab></person-group> (<year>2021</year>). DataIcfes. Available at: <ext-link xlink:href="https://www2.icfes.gov.co/data-icfes" ext-link-type="uri">https://www2.icfes.gov.co/data-icfes</ext-link> (Accessed June 6, 2021).</citation></ref>
<ref id="ref15"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Jayaprakash</surname> <given-names>S.</given-names></name> <name><surname>Krishnan</surname> <given-names>S.</given-names></name> <name><surname>V</surname> <given-names>J.</given-names></name></person-group> (<year>2020</year>). &#x201C;<article-title>Predicting students academic performance using an improved random Forest classifier</article-title>.&#x201D; in <source>2020 International Conference on Emerging Smart Computing and Informatics (ESCI)</source>. <publisher-loc>Pune, India</publisher-loc>. <fpage>238</fpage>&#x2013;<lpage>243</lpage>. doi: <pub-id pub-id-type="doi">10.1109/ESCI48226.2020.9167547</pub-id></citation></ref>
<ref id="ref17"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Kumar</surname> <given-names>V. U.</given-names></name> <name><surname>Krishna</surname> <given-names>A.</given-names></name> <name><surname>Neelakanteswara</surname> <given-names>P.</given-names></name> <name><surname>Basha</surname> <given-names>C. Z.</given-names></name></person-group> (<year>2020</year>). &#x201C;<article-title>Advanced prediction of performance of a student in an university using machine learning techniques</article-title>.&#x201D; in <source>2020 International Conference on Electronics and Sustainable Communication Systems (ICESC)</source>. <publisher-loc>Coimbatore, India</publisher-loc>. <fpage>121</fpage>&#x2013;<lpage>126</lpage>. doi: <pub-id pub-id-type="doi">10.1109/ICESC48915.2020.9155557</pub-id></citation></ref>
<ref id="ref18"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kuzilek</surname> <given-names>J.</given-names></name> <name><surname>Hlosta</surname> <given-names>M.</given-names></name> <name><surname>Zdrahal</surname> <given-names>Z.</given-names></name></person-group> (<year>2017</year>). <article-title>Open University learning analytics dataset</article-title>. <source>Sci. Data</source> <volume>4</volume>:<fpage>170171</fpage>. doi: <pub-id pub-id-type="doi">10.1038/sdata.2017.171</pub-id>, PMID: <pub-id pub-id-type="pmid">29182599</pub-id></citation></ref>
<ref id="ref20"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Liao</surname> <given-names>C.</given-names></name> <name><surname>Wu</surname> <given-names>J.</given-names></name></person-group> (<year>2022</year>). <article-title>Deploying multimodal learning analytics models to explore the impact of digital distraction and peer learning on student performance</article-title>. <source>Comput. Educ.</source> <volume>190</volume>:<fpage>104599</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.compedu.2022.104599</pub-id></citation></ref>
<ref id="ref21"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Liao</surname> <given-names>S. N.</given-names></name> <name><surname>Zingaro</surname> <given-names>D.</given-names></name> <name><surname>Thai</surname> <given-names>K.</given-names></name> <name><surname>Alvarado</surname> <given-names>C.</given-names></name> <name><surname>Griswold</surname> <given-names>W. G.</given-names></name> <name><surname>Porter</surname> <given-names>L.</given-names></name></person-group> (<year>2019</year>). <article-title>A robust machine learning technique to predict low-performing students</article-title>. <source>ACM Trans. Comput. Educ.</source> <volume>19</volume>, <fpage>1</fpage>&#x2013;<lpage>19</lpage>. doi: <pub-id pub-id-type="doi">10.1145/3277569</pub-id></citation></ref>
<ref id="ref22"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Ma</surname> <given-names>X.</given-names></name> <name><surname>Yang</surname> <given-names>Y.</given-names></name> <name><surname>Zhou</surname> <given-names>Z.</given-names></name></person-group> (<year>2018</year>). &#x201C;<article-title>Using machine learning algorithm to predict student pass rates in online education.</article-title>&#x201D; in <source>Proceedings of the 3rd International Conference on Multimedia Systems and Signal Processing - ICMSSP &#x2018;18</source>. <publisher-loc>New York, NY, USA</publisher-loc>: <publisher-name>Association for Computing Machinery</publisher-name>. <fpage>203</fpage>&#x2013;<lpage>207</lpage>. doi: <pub-id pub-id-type="doi">10.1145/3220162.3220188</pub-id></citation></ref>
<ref id="ref23"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>P&#x00E1;ez</surname> <given-names>A.</given-names></name> <name><surname>Guzm&#x00E1;n</surname> <given-names>D.</given-names></name></person-group> (<year>2018</year>). <article-title>Dise&#x00F1;o de un modelo para automatizar la predicci&#x00F3;n del rendimiento acad&#x00E9;mico en estudiantes del IPN</article-title>. <source>Revista Iberoamericana para la Investigaci&#x00F3;n y el Desarrollo Educativo</source> <volume>8</volume>, <fpage>246</fpage>&#x2013;<lpage>266</lpage>. doi: <pub-id pub-id-type="doi">10.23913/ride.v8i16.340</pub-id></citation></ref>
<ref id="ref24"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Rincon-Flores</surname> <given-names>E. G.</given-names></name> <name><surname>L&#x00F3;pez-Camacho</surname> <given-names>E.</given-names></name> <name><surname>Mena</surname> <given-names>J.</given-names></name> <name><surname>L&#x00F3;pez</surname> <given-names>O. O.</given-names></name></person-group> (<year>2020</year>). &#x201C;<article-title>Predicting academic performance with artificial intelligence (AI), a new tool for teachers and students.</article-title>&#x201D; in <source>2020 IEEE Global Engineering Education Conference (EDUCON)</source>. <publisher-loc>Porto, Portugal</publisher-loc>. <fpage>1049</fpage>&#x2013;<lpage>1054</lpage>. doi: <pub-id pub-id-type="doi">10.1109/EDUCON45650.2020.9125141</pub-id></citation></ref>
<ref id="ref25"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rodr&#x00ED;giez-Hern&#x00E1;ndez</surname> <given-names>C.</given-names></name> <name><surname>Musso</surname> <given-names>M.</given-names></name> <name><surname>Kyndt</surname> <given-names>E.</given-names></name> <name><surname>Cascallar</surname> <given-names>E.</given-names></name></person-group> (<year>2021</year>). <article-title>Artificial neural networks in academic performance prediction: systematic implementation and predictor evaluation</article-title>. <source>Comput. Educ. Artif. Intell.</source> <volume>2</volume>:<fpage>100018</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.caeai.2021.100018</pub-id></citation></ref>
<ref id="ref26"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Sekeroglu</surname> <given-names>B.</given-names></name> <name><surname>Dimililer</surname> <given-names>K.</given-names></name> <name><surname>Tuncal</surname> <given-names>K.</given-names></name></person-group> (<year>2019</year>). &#x201C;<article-title>Student performance prediction and classification using machine learning algorithms</article-title>.&#x201D; in <source>Proceedings of the 2019 8th International Conference on Educational and Information Technology&#x2014;ICEIT 2019</source>. <publisher-loc>New York, NY, USA</publisher-loc>: <publisher-name>Association for Computing Machinery</publisher-name>. <fpage>7</fpage>&#x2013;<lpage>11</lpage>. doi: <pub-id pub-id-type="doi">10.1145/3318396.3318419</pub-id></citation></ref>
<ref id="ref27"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll3">SINEACE</collab></person-group> (<year>2016</year>). <source>Modelo de Acreditaci&#x00F3;n para Programas de Estudios de Educaci&#x00F3;n Superior Universitaria</source>, <publisher-name>Lima</publisher-name>, <publisher-loc>Per&#x00FA;</publisher-loc>.</citation></ref>
<ref id="ref29"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tejedor</surname> <given-names>F.</given-names></name> <name><surname>Garc&#x00ED;a-Valc&#x00E1;rcel</surname> <given-names>A.</given-names></name></person-group> (<year>2007</year>). <article-title>Causas del bajo rendimiento del estudiante universitario (en opini&#x00F3;n de los profesores y alumnos). Propuestas de mejora en el marco del EEES</article-title>. <source>Revista de Educaci&#x00F3;n</source> <volume>342</volume>, <fpage>443</fpage>&#x2013;<lpage>473</lpage>. Available at: <ext-link xlink:href="https://dialnet.unirioja.es/servlet/articulo?codigo=2254218" ext-link-type="uri">https://dialnet.unirioja.es/servlet/articulo?codigo=2254218</ext-link> (Accessed May 6, 2021).</citation></ref>
<ref id="ref30"><citation citation-type="other"><person-group person-group-type="author"><collab id="coll4">UCI Machine Learning Repository</collab></person-group> (<year>2021</year>). Student Performance Data Set. Available at: <ext-link xlink:href="https://archive.ics.uci.edu/ml/datasets/student+performance" ext-link-type="uri">https://archive.ics.uci.edu/ml/datasets/student+performance</ext-link>.</citation></ref>
<ref id="ref31"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Viale</surname> <given-names>H.</given-names></name></person-group> (<year>2014</year>). <article-title>Una aproximaci&#x00F3;n te&#x00F3;rica a la deserci&#x00F3;n estudiantil</article-title>. <source>Revista Digital de Investigaci&#x00F3;n en Docencia Universitaria</source>. <volume>8</volume>, <fpage>59</fpage>&#x2013;<lpage>76</lpage>. doi: <pub-id pub-id-type="doi">10.19083/ridu.8.366</pub-id></citation></ref>
<ref id="ref32"><citation citation-type="other"><person-group person-group-type="author"><collab id="coll5">Vicerrectorado Acad&#x00E9;mico</collab></person-group> (<year>2017</year>). Cuando lo que se sabe nos dice cu&#x00E1;nto no se sabe&#x2014;Vicerrectorado Acad&#x00E9;mico. Available at: <ext-link xlink:href="https://vicerrectorado.pucp.edu.pe/academico/noticias/cuando-lo-que-se-sabe-nos-dice-cuanto-no-se-sabe/" ext-link-type="uri">https://vicerrectorado.pucp.edu.pe/academico/noticias/cuando-lo-que-se-sabe-nos-dice-cuanto-no-se-sabe/</ext-link> (Accessed May 3, 2021).</citation></ref>
<ref id="ref33"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Widyaningsih</surname> <given-names>Y.</given-names></name> <name><surname>Fitriani</surname> <given-names>N.</given-names></name> <name><surname>Sarwinda</surname> <given-names>D.</given-names></name></person-group> (<year>2019</year>). &#x201C;<article-title>A semi-supervised learning approach for predicting Student&#x2019;s performance: first-year students case study</article-title>.&#x201D; in <source>2019 12th International Conference on Information &#x0026; Communication Technology and System (ICTS)</source>. <publisher-loc>Surabaya, Indonesia</publisher-loc>. <fpage>291</fpage>&#x2013;<lpage>295</lpage>. doi: <pub-id pub-id-type="doi">10.1109/ICTS.2019.8850950</pub-id></citation></ref>
<ref id="ref35"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ya&#x011F;c&#x0131;</surname> <given-names>M.</given-names></name></person-group> (<year>2022</year>). <article-title>Educational data mining: prediction of students&#x2019; academic performance using machine learning algorithms</article-title>. <source>Smart Learn. Environ.</source> <volume>9</volume>, <volume>9</volume>:<fpage>11</fpage>. doi: <pub-id pub-id-type="doi">10.1186/s40561-022-00192-z</pub-id></citation></ref>
<ref id="ref36"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Yousafzai</surname> <given-names>B. K.</given-names></name> <name><surname>Khan</surname> <given-names>S. A.</given-names></name> <name><surname>Rahman</surname> <given-names>T.</given-names></name> <name><surname>Khan</surname> <given-names>I.</given-names></name> <name><surname>Ullah</surname> <given-names>I.</given-names></name> <name><surname>Ur Rehman</surname> <given-names>A.</given-names></name> <etal/></person-group>. (<year>2021</year>). <article-title>Student-Performulator: student academic performance using hybrid deep neural network</article-title>. <source>Sustainability</source> <volume>13</volume>:<fpage>9775</fpage>. doi: <pub-id pub-id-type="doi">10.3390/su13179775</pub-id></citation></ref>
</ref-list>
</back>
</article>