Over the last several years, technology has become an essential tool to support students and instructors in creating more effective educational experiences. In this context, the propagation of online learning environments (e.g., learning management systems, student diaries, library systems, digital repositories, and academic systems) has increased significantly, expanding the data generated about the educational process (Gaftandzhieva et al., 2020). These digital footprints can assist teaching and learning practices to foster better student achievement (Varanasi et al., 2018) and support teachers’ practices (Jivet et al., 2018).

To reach the potential of analysis of this data, learning analytics emerged as a field that focuses on collecting, analyzing, and reporting data about learners and contexts in which learning occurs (Siemens and Gasevic, 2012). The use of learning analytics can bring concrete benefits for students, teachers and institutions. The large amount of student data, such as demographic information, grades and student behaviors, expands the possibilities of retention strategies and academic success, thus moving away from leveling by the average, to meet the needs of each student in a personalized and data-oriented way (Tan et al., 2016; Aguerrebere et al., 2017).

Learning analytics has been widely researched and used in higher education institutions, especially due to the maturity level of adopting data analysis tools in these institutions (Leitner et al., 2017; Waheed et al., 2018; Charitopoulos et al., 2020). However, despite some promising results, learning analytics does not have the same level of adoption in other educational contexts, such as high schools (Cechinel et al., 2020; Ifenthaler, 2021). That is a limitation, as the adoption of educational technology in these levels of education has created environments where rich information could be extracted from the generated data (Schmid and Petko, 2019). For instance, Wastiau et al. (2013) performed a survey in Europe that demonstrated the importance of digital technologies in a middle school. Moreover, several recent papers are focused on the application of data analysis for high school students (Wastiau et al., 2013; Bernhardt, 2017; Martínez-Abad et al., 2020). While higher education institutions are quick to adopt learning analytic tools such as extensions of educational governance, teachers in high schools are often skeptical of the politics and utility of learning analytic tools, and often resist their implementation through their academic practice (Brown, 2020). Since the increased capacity of educational data mining has created a boost of educational technology tools development, Brown (2020) expressed the need to investigate how learning analytic tools shape activities beyond the classroom, and how they further influence curriculum and pedagogy.

There are many educational challenges in the high school context that involve all stakeholders in teaching and learning processes (Gaftandzhieva et al., 2020). Learning analytics can be used to address these challenges, such as school dropout (Khalil and Ebner, 2015), the difficulty of collaboration among students (Berland et al., 2015), the development of scientific argumentation and writing (Lee et al., 2019; Palermo and Wilson, 2020), and the development of computational thinking, which is an emerging ability for this age group (Grover et al., 2017). Teachers can be supported in understanding student practices and classroom variations (Quigley et al., 2017) and in monitoring students’ motivation levels (Aluja-Banet et al., 2019). Managers and decision-makers can use learning analytics to identify students who are in the vulnerable situation of not being able to graduate on time (Aguiar et al., 2015; Jiménez-Gómez et al., 2015) and in developing curricula that meet students’ needs and expectations (Monroy et al., 2013).

Based on this context and in the fact that several literature reviews present the potential in using learning analytics in different educational contexts, such as higher education, professional and workplace learning, vocational education, for massive open online courses, but not for the high school context, this paper presents a systematic literature review focusing on the applications of learning analytics 59 in high schools. The SLR enables the identification, evaluation, and interpretation of previous works that provide details about methods, tools and use of learning analytics in this context. More specifically, this review aims to provide a broad description of the main approaches, educational goals, techniques and challenges related to learning analytics and high schools.

The following sections of the paper present: section 2, a short background on learning analytics and previous literature reviews on the topic; section 3, the research questions investigated in this literature review; section 4, details about the method used; sections 5, 6, the results and the discussion of the finding in this study; finally, section 7, the limitations of the proposed literature review.

The most popular definition of learning analytics was presented by the Society for Learning Analytics Research (SoLAR) at the First Learning Analytics and Knowledge Conference in 2011—LAK′11 (Long et al., 2011). Learning analytics is defined as “the measurement, collection, analysis, and reporting of data about learners, learning environments and contexts to understand and optimize learning and their environments” (Siemens and Gasevic, 2012). Online learners leave behind data traces, and learning analytics can gather this data from different sources and learner activities, then analyze and provide meaningful insights and visualizations for institutional managers, teachers, and learners (Gedrimiene et al., 2020).

Despite the presumable advantages of using learning analytics, few publications explore the benefits of the learning analytics field in high schools (Ifenthaler, 2021). Although LA could address several challenges faced by high schools (e.g., student dropout and supporting the development of computational thinking abilities), it was not consistently used across different institutions (Charitopoulos et al., 2020; Ifenthaler, 2021). This fact could be a result of the lack of studies analyzing the context and the potential of LA for high schools, and the shortage in involving different stakeholders in the process of adoption of LA tools [as it is done in higher education (Maldonado-Mahauad et al., 2018; Tsai et al., 2018)]. Therefore, it is necessary to bridge the gap between technological capacity and tangible improvements in teaching-learning experiences. Given this context, it is very important to identify reports of experiences that allow knowing the practical consequences of the application of learning analytics (Slotta and Acosta, 2017).

Several literature reviews about learning analytics have been published in the last 11 years since the first edition of the LAK conference. For instance, Charitopoulos et al. (2020) synthesize the main methods and techniques adopted to support data analysis using learning analytics, based on papers published between 2010 and 2018. Waheed et al. (2018) presented a bibliometric analysis of the field in order to analyze publication counts, citation counts, co-authorship patterns, citation networks, and term co-occurrence. Among the main conclusions, the authors stated that higher education institution is a common keyword in the field.

Besides these general reviews, the most common topics of the previous SLR are related to specific methods, especially related to the development of visualizations and dashboards (Matcha et al., 2019b). In short, the main goal of these studies is to present current applications and tools to develop learning analytic visualizations, how students and instructors could benefit from learning analytic dashboards in practice, and the challenges and future research lines.

Previous studies also described how learning analytics developed in specific world regions. For instance, Cechinel et al. (2020) and Pontual Falcão et al. (2020) list several research initiatives and practical applications of learning analytics in Latin America. Similarly, Ferguson et al. (2015) described the perspectives of the adoption of learning analytics in Europe. All studies reported a major use in Higher Education Institutions in comparison to other levels of education.

Finally, other reviews report on the use of learning analytics in higher education (Leitner et al., 2017; Viberg et al., 2018), professional and workplace learning (Ruiz-Calleja et al., 2017; ?), and vocational education (Gedrimiene et al., 2020). However, to the best of our knowledge, there are no previous systematic literature reviews on adopting Learning Analytics at high schools, which is this paper’s main goal.

Moreover, it is important to remark that the high school context is significantly different from the educational settings of the previous reviews. For instance, the students in high school are usually under 18 years (different from higher education and professional learning), which could raise different ethical concerns and needs for LA. Besides, in general, the high school teachers’ technology backgrounds are not the same as professors at the universities. Finally, the data collected from students in high schools do not involve many interactions with learning management systems or MOOC platforms, which are the primary data collected for LA application.

The objective of this SLR was to identify primary studies that focus on the use of Learning Analytics techniques aiming at solving high school problems. Based on this context, this study addresses the following research questions:

The first research question focuses on the fact that the primary purpose of using Learning Analytics is educational rather than technological (Gašević et al., 2015). Therefore, this review starts by highlighting the educational motivation and problems that lead to the adoption of learning analytics in high schools. More specifically, we evaluated a subset of the categories proposed in previous works (Moissa et al., 2015; Viberg et al., 2018; Cechinel et al., 2020), such as predicting and enhancing students learning outcomes, analyzing students’ learning processes, supporting teachers’ decisions and reflection, and support writing activities. Subsequently, as the application of learning analytics in general means using data analysis, we intended to analyze the data processing approaches used in the learning analytics adoption process. Thus, our second research question was:

To answer the second research question, we adopted the categories proposed by previous works (Viberg et al., 2018): 1) Prediction: the use of regression and classification techniques to predict learning outcomes; 2) Clustering: application of different unsupervised methods to group similar instances of the data (e.g., students or learning material); 3) Relationship mining: this category includes methods related to association rule mining, sequential pattern mining, process mining, and casual data mining; 4) Distillation of data for human judgment: methods in this category include visualizations (e.g., dashboards) and statistical analysis to assist humans to make sense of the findings and support decision making; 5) Discovery with models: this category describes the application of models proposed in previous study, but analyzing new data to discover more patterns.

After understanding the approach of using learning analytics, we intended to identify the leading machine learning algorithms that have been implemented in the development of learning analytics systems for high schools. This analysis is essential as machine learning approaches are largely used by the LA community (Charitopoulos et al., 2020). Therefore, we intend to investigate the leading algorithms used and if they are aligned with the algorithms proposed for other educational contexts (Leitner et al., 2017; Ruiz-Calleja et al., 2017, 2021). As such, our third research question is:

In order to evaluate the potential of using LA for high school, the fourth research question focuses on describing the evidence of learning analytics research in high schools. Previous literature performed a similar evaluation for higher education success (Viberg et al., 2018). In this case, we analyzed if the selected papers offered evidence of a positive or negative impact of using learning analytics and if they presented (or not) empirical evaluation to support this evidence.

Finally, we also looked into the main challenges in using learning analytics reported by the studies retrieved in this literature review. This research question aims to make research aware of potential issues with the adoption of LA to high schools and provide a direction on how to avoid them. As such, the last research question is:

The SLR developed in this study followed the guidelines proposed by Kitchenham (Kitchenham, 2004). The review focused on the literature of the last 10 years, papers published between 2010 and 2020. The method adopted is composed of five steps: 1) definition of the research questions, 2) definition of the search strategies, 3) article selection process, 4) critical assessment, and 5) extraction of relevant fields. In the following sections, details about each step are presented.

This section presents the main insights of this systematic literature review, in the light of previous literature on learning analytics. Moreover, we highlight the main aspects that should be addressed by studies that adopt Learning Analytics in the high schools’ context.

The first research question analyzed in this study focused on the main educational goals for using learning analytics in schools. We found that these goals are focused on predictions of learning outcomes rather than supporting instructors and students in the decision-making process, or understanding students’ behavior. This is a characteristic of the initial articles in the field of learning analytics (Charitopoulos et al., 2020; Joksimović et al., 2019). Recent literature on learning analytics proposes the application related to a wide variety of goals that are not focused on prediction, such as providing feedback (Tsai et al., 2021b; Pardo et al., 2019), supporting counseling sessions (De Laet et al., 2020), analyzing students’ tactics and strategies (Matcha et al., 2019c,a), and understanding students’ knowledge construction in online discussions (Rolim et al., 2019; Ferreira et al., 2021; Neto et al., 2021).

Our review did not reveal papers suggesting the analysis of school context, which is considered a critical activity to ensure successful learning analytics adoption (Gasevic et al., 2019; Falcão et al., 2020; Tsai et al., 2021a). In general, the papers retrieved in this review did not include the stakeholders in the process of creating analytic models, tools, and systems, even the ones focusing on supporting teachers (Michailidis et al., 2017). The lack of understanding of the school context and the focus on prediction reinforce that the adoption of learning analytics for high schools is still taking its first steps.

Concerning the second research question, about the main technical approaches used in learning analytics for high schools, it is possible to draw a direct comparison of learning analytics for high schools and higher education. Viberg et al. (2018) found that in the context of higher education, predictive methods (including regression and classification) were the most frequent (32%), followed by relationship mining (24%) and distillation of data for human judgment (24%). In our analysis, the most important approach found was the distillation of data for human judgment, with 35.71%, followed by predictions (26.19%) and relationship mining (21.43%).

Although the top-3 categories coincide between the findings of the current review and the Viberg et al. (2018) review, the discrepancy between the number of papers related to predictive methods for higher education and the distillation of data for human judgment for high school is relevant. Two main factors may explain this result: 1) predictive methods applied in higher education, in general, are centered on the identification of aspects related to learning processes (Viberg et al., 2018), while the majority of the models for high schools focuses on learning outcomes (Jiménez-Gómez et al., 2015; Blasi, 2017); 2) the dashboards proposed by papers related to high schools (d’Anjou et al., 2019; Chen, 2020) are relatively simpler in comparison with those related to higher education (Matcha et al., 2019b). Based on these findings, it is possible to recommend that researchers developing LA for high schools should include more analysis related to the learning process to improve the quality of the dashboards to support students’ and teachers’ decision-making.

In terms of machine learning algorithms used in the studies included in the current review (third research question), almost 50% benefited from white-box algorithms such as decision trees and logistic regression. The use of white-box algorithms is a trend, and even a recommendation, in educational applications of machine learning (Almohammadi et al., 2017; Nistor and Hernández-Garcíac, 2018; Charitopoulos et al., 2020). Previous literature reviews in the context of higher education (Leitner et al., 2017) and workplace learning (Ruiz-Calleja et al., 2017, 2021) also indicate the trend of using different decision trees and regression algorithms to create models and perform analysis in the educational data. For instance, learning analytics methods rely on the fact that there is a person involved in the decision loop (Siemens and Gasevic, 2012), and white-box models provide more information about the decision made by the classifier.

The fourth research question revealed the lack of evidence on the success of the application of learning analytics reported in the studies included in the current review, suggesting that learning analytics is in the early days of adoption in high schools. The majority of the papers did not indicate any evidence demonstrating that learning analytics could be efficient when applied to practice. On the other hand, in the context of higher education, learning analytic applications are starting to scale to an institutional level (Viberg et al., 2018; Herodotou et al., 2020).

Another reflection of limited maturity in adopting learning analytics in high schools is the reduced number of challenges reported in the studies included in the current SLR (fifth research question). The central claims in this direction were related to data quality, privacy, and technical issues (e.g., internet connectivity and number of devices per classroom). While in the case of higher education institutions, the literature reported more complex concerns, such as stakeholders’ involvement, understanding of institutional needs, and more general ethical, privacy protection, and data governance issues (Gasevic et al., 2019; Cechinel et al., 2020).

Based on the results described in this paper, we highlight the following aspects to be considered by researchers working with learning analytics in high schools:

1. Institutional diagnosis: A very important issue we have noticed in the papers included in the SLR is the absence of a methodological process to understand schools needs and context for learning analytics adoption. In this direction, several frameworks have been proposed for adoption of learning analytics in higher education. For instance, Tsai et al. (2018) proposed SHEILA, a framework that guides higher education institutions in adoption of learning analytics by providing relevant instruments for involvement of relevant stakeholders and by documenting actions taken, policy questions, and challenges commonly experienced by institutions. SHEILA has widely been used across the world (Maldonado-Mahauad et al., 2018; Broos et al., 2020; Falcão et al., 2020; Vigentini et al., 2020)). Therefore, SHEILA or similar frameworks could be used as a good starting point to understand the needs of high schools regarding learning analytics.

The primary limitation of this study is related to the search process in which we only focused on papers that contain the specific keyword “high school”. This could potentially exclude papers that describe the adoption of learning analytics in secondary schools, the terminology used by other countries to refer to high school in their educational systems.

Secondly, a few papers had limited information about the methods and techniques used, which led to several coding some studies with labels such as “no details” and “no evidence” in some of the categories that were analyzed in the current review. We decided to keep these papers nevertheless because they contained relevant information to at least one research question. However, it is important to highlight that we performed a critical assessment to assure the quality of the paper.

Finally, this review did not focus on papers in the fields that are related to learning analytics such as educational data mining and artificial intelligence in education, which could broaden the reach of this SLR.