Systematic Review ARTICLE
Flipped Classroom in Teacher Education: A Scoping Review
- Department of Teacher Education, Norwegian University of Science and Technology, Trondheim, Norway
Although flipped classroom (FC) has been popular in education since the 2000s, there is a lack of reviews on how the teaching approach has been applied and what has been gained in the field of teacher education. Most reviews focus either on implementation and learning outcomes with students in higher education and disciplines other than education or on the latter approaches with primary and secondary school pupils. This article presents a scoping literature review of 33 studies published between 2014 until 2019 on flipped classroom (FC) in teacher education. Our analysis points out that studies were mainly conducted in the United States, with an increased implementation in European and Asian countries, and with adoption primarily in the disciplines pedagogy, science, and language arts. Moreover, a majority of studies employed mixed methods with surveys being the most commonly used instrument to collect data. Two main foci were identified across the reviewed studies: student perceptions and academic performance. The analysis of the former revealed six outcomes (1. Attitude, motivation, and emotion; 2. Content delivery; 3. Learning environment; 4. Learning experience; 5. Instructor and student presence; 6. Engagement). Based on our synthesis, we discuss current trends and future development in the research field, FC's pedagogical value added in teacher education, and potential knowledge gaps in the research literature.
Recent improvements in and access to digital technologies have paved the way for flipped classroom (FC) as an appealing, innovative, and motivating pedagogical teaching approach (Abeysekera and Dawson, 2015). Compared to traditional lectures often characterized by passive and transmissive modes of teaching, FC can be viewed as a “student-centered approach to teaching and learning that emphasizes student engagement and active learning” (Steen-Utheim and Foldnes, 2018, p. 308). Despite being around since the 2000s (cf. Baker, 2000; Lage et al., 2000), FC gained popularity with Bergmann and Sams (Bergmann and Sams, 2009, Bergmann and Sams, 2012) who were concerned about high school students missing end-of-day classes.
Bergmann and Sams (Bergmann and Sams, 2012, p. 13) state that the basic concept of FC is “that which is traditionally done in class is now done at home, and that which is traditionally done as homework is now completed in class.” Building on the former definition, Bishop and Verleger (2013, p. 5) underline two aspects of FC: (1) “interactive group learning activities inside the classroom” and (2) “direct computer-based individual instruction outside the classroom.” Similarly, Abeysekera and Dawson (2015, p. 3) note that in FC, “learning activities that are active and social” occur inside the classroom while “most information-transmission teaching” occurs outside the classroom.
The consequent enthusiasm among practitioners and researchers has resulted in increased interest in FC as an educational research area, in turn resulting in several literature reviews and meta-analyses (O'Flaherty et al., 2015; Betihavas et al., 2016; DeLozier and Rhodes, 2017; Akçayir and Akçayir, 2018; Hew and Lo, 2018; Turan and Akdag-Cimen, 2019). However, reviews on FC rarely focus on teacher education with study populations including teacher educators, pre-service teachers, or mentor teachers. O'Flaherty et al. (2015), for example, investigated FC in higher education. Their review included studies with empirical evidence from across subject disciplines (e.g., nursing, business management, social science) and found indirect evidence for FC improving academic performance, and student and staff satisfaction. However, most of the included studies were not related to teacher education and thus the review has limited utilization to the field. In addition, Hew and Lo (2018) conducted a meta-analysis on FC and student learning in health profession education, while Turan and Akdag-Cimen (2019) focused on FC and English language teaching.
Although there is a growing interest in the field, little is known about the use of FC in teacher education, where there is currently a lack of reviews systematically organizing empirical studies in the research field. Therefore, it is necessary to conduct a scoping review of FC in teacher education to inform researchers and practitioners about the latest developments, knowledge, experiences, and research foci in the field. The scoping review approach has become growingly popular among researchers for synthesizing research data (Davis et al., 2009; Daudt et al., 2013) because a scoping review can, according to Arksey and O'Malley (2005, p. 21) help meet four goals: “(1) To examine the extent, range and nature of research activity; (2) To determine the value of undertaking a full systematic review; (3) To summarize and disseminate research findings; (4) To identify research gaps in the existing literature.” Thus, the current study adopts the scoping review approach and seeks to obtain both quantitative and qualitative data which will help to address the abovementioned information for researchers and practitioners. In the next sections of the article, we introduce the review method, followed by a presentation of results and discussion of findings.
provides a preliminary assessment of the potential size and scope of available research literature. It aims to identify the nature and extent of research evidence (usually including ongoing research). […] Scoping reviews are able to inform policymakers as to whether a full systematic review is needed. They share several characteristics of the systematic review in attempting to be systematic, transparent and replicable.
The present study adopted the five-stage framework of Arksey and O'Malley (2005, p. 22): (1) identifying the research question, (2) identifying relevant studies, (3) study selection, (4) charting the data, and (5) collating, summarizing, and reporting the results.
Identifying the Research Questions
The following two research questions (RQs) guided the review:
1. What are the trends in FC in teacher education?
2. What are the research foci and findings of the presented studies on FC in teacher education?
The first question aims to provide an overview and a map of trends in FC in teacher education by providing details about variables including author details, year of publication, country location, subject disciplines, research methodology, study design, and participants. The second research question aims to summarize and disseminate the different approaches to and outcomes of FC in teacher education.
Identifying Relevant Studies
The search terms were developed and categorized based on two dimensions according to the purpose of the review. One dimension was related to FC (i.e., the activity examined), while the other dimension was related to pre-service teachers (i.e., the participants in the activity examined) to narrow the search within the field of teacher education. Each search term was separated by the Boolean OR operator and each dimension was separated by the Boolean AND operator, which was outlined in Table 1.
A set of inclusion and exclusion criteria were developed to focus the scope of the review (Table 2). Based on previous research, FC has been observed in the research literature since the early 2000s (Lage et al., 2000), and so the last 20 years were set as the time period for the review. Other criteria include peer-reviewed articles, English and Norwegian languages, a focus on empirically driven studies, and with populations related to teacher education where the sample size is explicitly mentioned.
Two electronic databases were searched: Education Resources Information Center (ERIC) and Web of Science. Moreover, a manual search or “hand-searching” (Chapman et al., 2010, p. 23) reference lists was conducted after searching the databases to “locate relevant studies missing in the database searches” (Røkenes and Krumsvik, 2014, p. 255).
Based on the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement (Moher et al., 2009), Figure 1 shows the study selection process, including procedures for searching databases, searching manually, screening titles and abstracts, screening full texts, and selecting eligible articles for inclusion. The last database search was conducted on January 1, 2020; 92 articles were identified, from among which 17 duplicates were removed. All the potentially relevant articles went through a two-step screening process. The first step was to exclude irrelevant articles by screening titles and abstracts. The second step was to filter out unrelated articles by screening full texts. After the first step, 12 articles from ERIC and 19 from Web of Science were excluded because they were unrelated to FC or not conducted in the field of teacher education. After the second step, 11 articles from ERIC and two from Web of Science were removed mainly due to research focus and sampling, such as focusing on the effects of an online course instead of FC or the unclear number of study participants. Eventually, 33 articles were included for further analysis, 22 from ERIC, nine from Web of Science, and two found through a manual search. Full texts were obtained of the 33 studies, and each study was reviewed and confirmed as suitable for inclusion by the authors.
Figure 1. Flowchart diagram of the screening and selection procedure (Moher et al., 2009, p. 267).
Data Charting and Collation
Summaries of each study were developed based on indicators including authors, year of publication, country location, research design, methodology, study population and sample size, and brief descriptions of outcomes (Table 3). In Table 3, EG (experimental group) has taken courses in FC and CG (control group) has taken courses in a traditional classroom. A detailed table with study title, discipline, research questions, and characteristics of participants can be found in the Supplementary Material). Next, studies were analyzed using a coding and categorization strategy (Saldaña, 2016).
Summarizing and Reporting Findings
In accordance to the fifth stage of Arksey and O'Malley's (2005) framework for scoping reviews, the next sections summarize, report on, and discuss findings from the 33 included studies.
Results and Discussion of Findings
The coding and analyzing system included three main categories (i.e., general characteristics, research methods, and research foci), each with several subcategories.
General Characteristics of Included Studies
Distribution by Year of Publication
The results from the database searches revealed that the first studies about FC in teacher education were published in 2014, while according to a previous review article the first study about FC was published in 2000 (Akçayir and Akçayir, 2018). Thus, FC research in teacher education started nearly 14 years after the first published study on FC. As shown in Figure 2, even though FC research in teacher education started late, according to Akçayir and Akçayir, this is still within the main tendency in the FC research: “After more than a decade, flipped classroom studies became popular among scholars; the numbers of such studies began to steadily increase after 2012” (2018, p. 337).
Distribution by Country
Figure 3 shows that nearly one third of the reviewed studies were conducted in the USA (10 of 33 articles), while seven were conducted in Turkey. Six studies were conducted in Spain; two were conducted in each of Australia, South Africa, and South Korea; and one was conducted in each of Canada, China, Kuwait, and Norway.
Distribution by Subject Discipline
Teacher education involves many subject disciplines, such as general education, mathematics, science, language, history, and health science. In this review, articles were categorized as “pedagogy” when courses were not aiming at a specific subject discipline or a certain group of student teachers, such as an introduction to educational psychology course, an instructional methods course, and an educational technology course. Figure 4 shows that nearly one third of the reviewed studies (10 articles out of 33) were in pedagogy. When further subdividing the studies into different subject disciplines, studies on FC in teacher education were mainly in the science (six articles), language (five articles), and mathematics (three articles) disciplines.
Research Methods of Included Studies
Out of the 33 reviewed studies, more than half (17) did not explicitly identify their methodological paradigm. Therefore, we categorized the methodological paradigms in these articles based on the description in the method sections. Figure 5 shows that 22 (67%) of the studies used mixed methods for data collection, 8 (24%) used quantitative methods, and 3 (9%) used qualitative methods.
Participants Involved in the Research
The current study focuses on the FC in teacher education; therefore, the authors were interested in the perceptions of pre-service teachers and the effects (e.g., academic, engagement, and emotional) on pre-service teachers that were explored in all the reviewed studies. Figure 6 shows that participants who were taught using a FC approach were coded as the experimental group (EG), while those who were not were coded as the control group (CG). Of the 33 reviewed studies, 18 were conducted only with participants who had learned using an FC approach; 14 were conducted with participants who learned using both an FC and a traditional approach. Only one study (Erdogan and Akbaba, 2018) involved participants who had not taken any courses taught using the FC approach; this study investigated student teachers' opinions about whether the social studies classroom should be flipped.
The number of EG members ranged from 14 to 371, while the number of CG members ranged from 9 to 162. The total number of participants across all 33 reviewed studies ranged from 9 to 371.
Instruments for Data Collection
Table 4 lists all instruments used for data collection in the reviewed studies; S + [number] represents the studies reviewed (e.g., S1 represents the first study in the reviewed list). Surveys, both paper-based and online, were the most commonly used tool (over two thirds) to explore participants' perceptions. Some studies applied both pre- and post-surveys (Fraga and Harmon, 2014; Dove and Dove, 2017a,b; Hall, 2018). Others conducted surveys after an FC intervention. Test scores were frequently used to collect data about students' academic performance, and scores could come from final exams, tests, or quizzes. Interviews were often used to collect qualitative data, and focus-group interviews were used more frequently than one-to-one interviews. Due to the characteristics of teacher education, lesson plans made by student teachers could also be considered an instrument. Classroom observation was conducted in FCs as well, with the observations in the form of both video-taped lessons and field notes. Student teachers' weekly journals, teacher educators' journals, and student teachers' feedback were all reflections of participants in teaching and learning and could be generalized as self-report materials. Tasks completed by student teachers also served as a tool to collect data. As a teaching methodology or pedagogical instructional model closely related with information and communication technology (ICT), the FC is always linked with computers or the Internet. Therefore, a course blog and student teachers' log information could also be considered data. Furthermore, course assessments are often conducted at the end of a university-level course, and so that data can also be used to study the FC.
Research Foci of Included Studies
Most of the reviewed studies (27) seemed to be concerned with students' perceptions of the FC, and some (14) also explored whether the FC could improve students' academic performance (Figure 7). Other aspects examined were teacher educator's perceptions of the FC (two studies), students' self-regulated learning (SLR; two studies), students' anxiety (two studies), students' self-efficacy (two studies), and students' self-perceptions of pedagogical knowledge, technological knowledge, and technological pedagogical content knowledge (TPACK; one study).
Student Perceptions of the Flipped Classroom
The top research focus reflected in 27 of the 33 articles is student perceptions of the FC teaching approach. Figure 8 gives an overview of the aspects of student perceptions covered in those 27 studies.
Table 5 lists details about the different research foci and their representative articles. Each aspect will be elaborated in the following text.
Attitude, Motivation, and Emotion
The reviewed studies were categorized as examining attitude, motivation, and emotion when they explored student teachers' or teacher educators' attitudes, motivations, and emotions vis-à-vis the FC. Students had favorable attitudes and “a general positive opinion” (González-Gómez et al., 2016, p. 456, 458). For example, on flipped classes, students said “it was a really good class” (Conner et al., 2014b, p. 74) and “this class was by far the best class we have taken” (Conner et al., 2014b, p. 74, 75). Student participants in Fraga and Harmon (2014) stated they liked the FC model because of “the time flexibility” (p. 22, 24) and “being in control of their own individual learning” (p. 22, 24). Survey data from García-Sánchez and Santos-Espino (2017) showed that students were satisfied with the FC approach due to “remarkable facts that suggest participants were pleased with their creative abilities” (p. 178). Focus-group interview data from Kurt (2017) indicated that all participants were “highly satisfied” (p. 216) because of “their perception of better learning and their enjoyment of the flipped class model” (p. 216). Sayeski et al. (2015) compared three instructional conditions (independent, facilitated, and flipped) and concluded from students' survey data that “the most popular condition was the flipped classroom condition” (p. 302). In Ng's study (2018), eight students attended the focus-group interview and “they all liked” (p. 72) the FC approach and its advantages of “promoting self-learning, innovation, and flexibility” (p. 72). FC was associated with “a general positive perception” (Jeong et al., 2018, p. 163) not only in the face-to-face learning environment but also in the online environment; 91% of participants in van Wyk's (2018a) study strongly agreed that FC was a student-centered approach that increased student active learning and promoted and increased classroom interaction.
However, students' attitudes toward the FC compared to the TC were sometimes “variable” (Tomas et al., 2019, p. 12). A student in Conner et al. (2014a) “felt the flipped classroom approach went well, but was challenging to keep up with if you forgot to watch the online videos prior to attending class” (p. 73). One student commented that in FCs students should “take responsibility for not only our own learning, but for our classmates as well” (Graziano, 2017, p. 124). A student in Dove and Dove (2017a) also said that the responsibility of learning was “completely on the students” (p. 138). Besides responsibility, the responses of students who disliked the FC model “fell into two categories—issues of time management and confusion” (Fraga and Harmon, 2014, p. 22). Tomas et al. (2019, p. 12) summarized students' attitudes toward FCs, stating “In spite of the variability in students' perceptions, the majority appeared to respond positively to the flipped classroom approach.” Furthermore, student teachers seemed to favor having flipped classes in the future. Survey data from Jeong et al. (2016) showed that 94% of participants agreed or strongly agreed that they were “willing to have more courses flipped” (p. 753). In Kurt (2017), one student said, “I believe my future students will benefit from” (p. 217) flipping a language classroom. A participant in van Wyk (2018b) posted on the class blog that “I love flipping my class strategy and will definitely use it” (p. 19). There were other varying opinions as well. For example, survey data from Jeong et al. (2019a) revealed that students “agreed or strongly agreed to have more courses flipped” (p. 8) because the flipped course was “interactive” (p. 10) and “interesting” (p. 10). Survey data from Graziano (2017) also showed that “the majority of students said very likely or likely” (p. 125) in response to being asked about flipping their own classrooms in the future. One student teacher commented in the informal discussion that “I will not have the time during my first few years of teaching to accurately gather or make videos on my own” (p. 124).
Several studies also explored students' motivation vis-à-vis FCs. Sengel (2016)'s survey data revealed that students in FCs “were willing to pay more” (p. 495) for the course and “were more motivated” (p. 495). Survey data from Tomas et al. (2019) indicated that “for a majority of students, the flipped approach enhanced their motivation to learn” (p. 12). Turan and Goktas (2018) focused their research on the impact of FCs on students' motivation; their survey data revealed that “the motivation for students in the experimental group was greater than that of the control group” (p. 142), which meant that students' motivation in FCs was greater than that in TCs. Turan and Goktas (2018) categorized motivation within the scope of ARCS theory, which identifies attention, relevance, confidence, and satisfaction as four critical components that affect motivation. They discovered that “the attention level of students [in the experimental group, our interpretation] toward the course was clearly high” (p. 142). In fact, all the ARCS scores in Turan and Goktas (2018) were higher for the EG than for the CG. Turan and Goktas (2018) also found that the leading positive motivational factor was hands-on activities, while the leading negative motivational factor was difficulty in flipping. However, the survey data of Yough et al. (2017) revealed that “all analyses were non-significant with the exception of two motivation outcomes” (p. 6); even “preservice teachers in the traditional sections were more likely to report greater levels of intrinsic motivation […] and identification […] than those in flipped sections” (p. 6). Cabi (2018) also found that some students were not motivated because they “did not want to put an effort on it” (p. 214) and felt that “the topics were boring and unnecessary” (p. 214).
Jeong et al. (2016, 2018) explored students' emotions and self-evaluations after participating in FCs. Both studies had the same findings regarding positive and negative emotions: “the overall scores were very high in positive emotions” (2018, p. 7), with feelings of fun and enthusiasm having the highest score. Regarding negative emotions, boredom had the lowest score and concern had the highest. These findings indicated that many students agreed that the FC model was fun and that feelings of concern and nervousness had brought more negative emotions to them than boredom. In their recent study, Jeong et al. (2019a) concluded from survey data that “positive emotions were highly scored” (p. 11) by students in an FC compared to those in a TC.
As one of the characteristics of FCs, content delivery differs from that in TCs where usually a lecturer presents his or her lecture during class time. In an FC, a lecturer normally records the lecture in advance and students access the content outside class time with different equipment. Sengel (2016) investigated equipment to access online videos and found that the most commonly used equipment was “laptop[s] (95%) and cellphone[s] (80%)” (p. 492), while “IPod[sic] touch (27%) and IPad[sic] (36%) were the equipment” least used (p. 492). In addition, many researchers explored students' perceptions of content delivery and found they had both positive and negative opinions.
Some students thought that content delivered before class, such as narrated presentations with “talking heads” and online lectures, were “convenient, effective and engaging” (Adnan, 2017, p. 215), “beneficial” (Conner et al., 2014b, p. 73), and “easy to watch and easy to follow” (Conner et al., 2014a, p. 71). Other benefits included having “the opportunity to pause” (Conner et al., 2014a, p. 71) and being able to “wait, stop, go back” (Dove and Dove, 2017b, p. 325) and to “stop and replay” (Tomas et al., 2019, p. 12). This approach also made students “come to the class prepared” (Cabi, 2018, pp. 213-214). A student teacher in Dove and Dove (2017b) stated that “PowerPoints […] brings you step by step about what is happening” (p. 325). Survey data from Graziano (2017) revealed that students “enjoyed the flexibility in delivering course content” (p. 124). Students in Jeong et al. (2016) had “a general positive opinion” (p. 752) about utilizing video lectures, stating they were of “great help to achieve learning goals” (p. 752). This sentiment was echoed by participants in Kurt (2017) who stated that videos helped them “learn the material better” and made the material “more enjoyable” (p. 216). According to Lee and Bonk (2019), 23 of 27 participants reported that “class preparation through online video lectures helped them pay attention to their f2f classes” (p. 16) and “encouraged them to participate more actively in the group discussions” (p. 16). Jeong et al. (2019a)'s survey data showed that “students agreed or strongly agreed that having video lectures or other “flipped” materials before class helped or help to achieve the learning goals and to complete the in-class activities more confidently or in an easier manner” (p. 7). The researchers noticed that “students attended class after revising the flipped materials and were more willing to participate in student-centered activities such as collaborative chores” (p. 7). Tomas et al. (2019) found that 98.8% of students believed that “viewing the flipped videos helped them to understand the key concepts” (p. 11) in their course. According to the survey data in Helgevold and Moen (2015)'s study, most participants “found the online lectures to be a useful learning arena” (p. 35) because they could control “time, placement and pace” (p. 35). Similarly, Kurt (2017)'s focus-group interview data showed that participants “all appreciated watching the lectures at any time they wanted and being able to decide on their own pacing. They paused, rewound and replayed the video lectures as they needed” (p. 217).
Meanwhile, some students encountered “technological issues” (Conner et al., 2014a, p. 72) and “lost their attention” (Adnan, 2017, p. 215) while watching the videos or narrated presentations. Some thought the content was “difficult” or “not sufficient” (Cabi, 2018, p. 214). In Conner et al. (2014b), one student teacher stated that “honestly they could have just given us the handouts and I would have gotten as much from it” (pp. 72-73). Students in the Graziano (2017) study acknowledged they were not “‘tech savvy’ or comfortable with technology” (p. 126); furthermore, making videos was “time consuming” (p. 124, 126) for teachers. In Ng's study (2018, pp. 72-73), student teachers mentioned difficulties they encountered when watching online videos: (1) “they did not have a teacher to ask when they could not understand the online video”; (2) “they had to wait until the following week to ask the teacher when they could not understand a concept”; and (3) “they might forget about asking the questions.” These difficulties were also encountered by participants in Sengel's study (2016), where “70% of them indicated that they did not have chance to ask for these problems someone outside of the class time” (p. 493). Accordingly, Ng (2018) suggested that when utilizing an FC teaching approach, teacher educators could suggest student teachers email or contact their teacher educators “if they encounter any difficulties prior to the next lesson” (p. 73). Sammel et al. (2018) found that “the median number of online videos watched by the participants was only four out of eight” (p. 55), and students were not well-engaged even though they “knew that 50% of their final mark was derived from the video content” (p. 55). Sammel et al. (2018) also discovered that even though students highlighted convenience in terms of time management and repeatability, they still preferred face-to-face lectures and believed it was what they paid for as campus students.
One student in Conner et al. (2014b, p. 73) said “Don't just read me what is on the PowerPoint. We are in college, we can read.” Other student teachers in Conner et al. (2014b, p. 73) suggested “providing a set of partially completed notes for future students because that would encourage the students to watch the online videos to ‘fill in the blanks in your notes”’. In Ford (2015), the teacher educator attempted to use two formats to record videos—teach an imaginary classroom and create large slides or storyboards. García-Sánchez and Santos-Espino (2017) found that student teachers preferred “a lecture displayed as some variant of slideshow, mostly combined with narrator's face and voice” (p. 176) and “simple record-and-publish software tools” (p. 176) rather than “more sophisticated settings” (p. 176) where instructors “used puppets and cartoons as characters” (p. 176). Furthermore, the length of a video lecture with the FC approach was explored in some of the reviewed studies. For example, Wagner et al. (2013) suggested not making videos longer than 10 min because very few college students (35.7%) reported enjoying watching long videos.
Some teachers provided online lectures and quizzes as out-of-class activities. For example, González-Gómez et al. (2016) provided two types of online quizzes, “multiple-choice online quizzes about the contents taught in the video lessons that students had to complete after watching or/and reading the proposed material” (p. 456) and “questionnaires inserted in the video lessons” (p. 456). In these cases, students had to provide the right answer to be able to watch the remaining part of the video. The point was to encourage the students to watch the complete video lessons. Jeong et al. (2016) found that online quizzes were “useful to point out and overcome the most complex contents” (p. 753). In González-Gómez et al. (2016), 87 of 101 participants agreed or strongly agreed the online quizzes “provided together with the multimedia material had allowed them to achieve the learning objectives” (p. 457), and “nearly 90% of students were able to point out the most complex contents before the class and therefore to focus to overcome them after completing the online quizzes provided with the video lessons” (pp. 457-458). However, students in Conner et al. (2014b)'s study argued that online quizzes “were not very challenging and that being allowed to use notes during the quiz was pointless” (p. 73) and it was “very easy to forget” (p. 73).
In the review, learning environment can be understood as focusing on psychosocial factors in the classroom. Here, the FC was regarded as “flexible,” “stress free” (Adnan, 2017, p. 216), and potentially improving “the confidence” (Conner et al., 2014a, p. 72) of students. Karaaslan and Çelebi (2017)'s survey data indicated “the students generally had positive attitudes” (p. 652) regarding the learning environment. Participants in Kurt (2017)'s study also had “positive perceptions of their experiences in the flipped learning environment” (p. 216); they mentioned that the learning environment was “student-centered, more positive and less stressful” (p. 217). However, some students in Conner et al. (2014a)'s study had the opposite opinion, stating the FC was “a waste of time and did not contribute to learning” (p. 73).
Graziano (2017) found that the learning environment was more interactive; one student said, “there is more student–teacher interaction in a flipped class” (p. 124). According to Jeong et al. (2016), over 91% participants “thought that the course was more interactive than other courses taken in the same school” (p. 753). By analyzing video-taped classroom observations, Dove and Dove (2017b) found that FCs (flipped both with teacher's videos and lecture videos from Khan Academy) “were able to incorporate more opportunities for interaction and communication within and between students and the instructor” (p. 328). In van Wyk (2018b)'s study, a participant said on the blog that “The flipped method helps me to collaborate with my classmates on specific tasks and establish positive relationships with others” (p. 22).
The reviewed articles were coded for learning experience if they explored participants' experiences with FC both inside and outside classrooms. Students in Adnan (2017)'s study talked about their learning experience with FC, saying they believe FC can enhance learning because it encourages “social classroom learning” (p. 218) and they can learn “from each other in the classroom” (p. 218). Students in Jeong et al. (2019a)'s study considered the FC course “a valuable learning experience” (p. 10) and “significantly more interactive” (p. 7), saying it “provided a higher perception about the learning process for the same contents” (p. 7). Survey data in Lee and Bonk (2019)'s study revealed that the flipped class integrated with team-based learning can offer students “much higher and richer learning experiences” (p. 17). The findings of the reviewed articles revealed that students connected their learning experiences with their responsibility. For example, students in Dove and Dove (2017b) felt they were “encouraged” (p. 325) to take responsibility for their own learning, and they “positively reacted to the increased opportunities for structured small group activities” (p. 325). Similarly, student teachers in van Wyk (2018b)'s study stated that FC “created a positive lived experience” (p. 13) and forced them to “take responsibility for their own learning” (p. 13). According to van Wyk (2018a), 97% of student teachers agreed that FC pedagogy forced them to “take responsibility for their own learning” (p. 260). Lee and Bonk (2019) found that because students were forced to take responsibility, “there is a significant difference of learning time” (p. 16) in FCs compared to TCs and that students spent more time in FCs.
Instructor and Student Presence
Both instructors' classroom role and guiding role were emphasized in Adnan (2017), as instructors were in the classroom “interacting, answering questions and providing help” (p. 216). Dove and Dove (2017b)'s survey and interview data revealed that “students were overwhelmingly positive about the role of the instructor in their learning” (p. 324). In van Wyk's study (2018a), participants perceived that instructors played “an important role in creating a positive lived experience” (p. 263) in open-distance e-learning environments. Students in Lee and Bonk (2019)'s study reported they interacted “more frequently” (p. 18) with instructors in FCs and “received more frequent feedback on their learning” (p. 18) from instructors. In addition, relationships with instructors seemed more positive in FCs; for example, 81.4% of participants in Lee and Bonk (2019)'s study reported that instructors were “accessible and very helpful” (p. 18). The teacher educator in Ford (2015)'s study obtained a high rating on teaching evaluations. Not only were relationships with instructors improved but relationships with peers were “quite close and comfortable” (p. 19), as a student in the Lee and Bonk (2019) study stated. Participants in van Wyk (2018b)'s study agreed that “the teacher played an important role when it came to decision making about what out-of-class and in-class activities should be carefully integrated in order for students to understand the strategy and be motivated to prepare for the class” (p. 13). However, they “were less in agreement … concerning the role that teachers play in moving around and assisting students who are struggling while at the same time correcting misconceptions and providing one-on-one tutoring” (p. 13). In addition to instructor presence, Adnan (2017) examined student presence and found that “almost all students mentioned their adaptation to new roles/competencies” (p. 217) for the flipped course. The students had been educated in TCs since they started school, so it would naturally take some time for them to adapt to the new teaching approach.
Regarding students' engagement in learning, the reviewed studies had generally positive results and included only a few negative opinions. The survey findings of Tomas et al. (2019) painted “an encouraging picture of students' engagement with the flipped videos” (p. 9) and reflected that “the majority of students watched the flipped videos more than once, generally prior to attending class” (p. 9). However, the teacher educator in Ford (2015)'s study found through classroom observations that “not all students were watching the videos” (p. 375). Analyses of the empirical data in Helgevold and Moen (2015) showed that “this flipped classroom model, to some extent, seems to have stimulated students' participation and engagement” (p. 40). Students in Lee and Bonk (2019)'s study reported they were “immersed into learning in class time” (p. 20) and “never dozed at all” (p. 20) in FCs. By analyzing data from the Teachers' Sense of Efficacy Scale (TSES) developed by Tschannen-Moran and Hoy (2001), Kurt (2017) found “there was a significant difference between the experimental and control groups in their gain scores … and the subscales of student engagement and classroom management” (p. 215), indicating the differences in students' engagement in FCs compared to TCs were statistically significant in favor of FCs.
Academic Performance in a Flipped Classroom
Whether FCs can influence students' learning outcomes was one of the research foci in nearly half the reviewed studies (14 of 33) that examined students' academic performance in FCs. Table 6 lists statistical results about students' academic performance in FCs and the representative articles.
Of the 14 studies, four (Choi and Lee, 2015; González-Gómez et al., 2016; Kurt, 2017; Jeong et al., 2018) found a significant difference between the EG and the CG in favor of the EG. Kurt (2017) found the EG “overperformed” (p. 216) during final exams compared to the CG and that “the difference was statistically significant” (p. 216). The statistical results from Jeong et al. (2018)'s study “demonstrated that the flipped-classroom model gave better outcomes than previous classes not applying the flipped-classroom model” (p. 8), as “significant differences at the 95% significance level (p < 0.05) were observed” (p. 6) in favor of the FC. Choi and Lee (2015) found that the FC approach was more effective for students to learn knowledge and skills for instructional material production and that the effects were more observable for a difficult task.
Yough et al. (2017) and van Wyk (2018b) found no statistically significant differences in learning outcomes between the EG and the CG. However, these two studies had similar empirical findings to van Wyk (2018a) in that they found the FC teaching approach “enhanced” (van Wyk, 2018a, p. 262) students' academic achievement. Yough et al. (2017)'s statistical analysis revealed that “participants in the flipped sections had increased meaningful learning outcomes over participants in the traditional sections” (p. 6). “Preservice teachers in the flipped sections were more likely to score higher on items related to topics of information processing, development, and motivation compared with traditional section students” (Yough et al., 2017, p. 6). In comparing student teachers' pretest and posttest results, van Wyk (2018b) found that FC pedagogy is “a more effective digital pedagogical tool” (p. 12) and that the EG “outperformed the control group […] in terms of the final examination score” (p. 12). Furthermore, one of the participants in van Wyk (2018b)'s study stated “the flipped class strategy made it easier for me to learn better [.…] I am a visual learner. The videos help to better my understanding and increased my results in the course […] I increased my examination final mark to distinction (83%)” (p. 21).
Contrary to the above studies, three studies (Fraga and Harmon, 2014; Sengel, 2016; Cabi, 2018) found no significant difference in learning outcomes between the EG and the CG. Based on a statistical analysis, Sengel (2016) found “the flipped classroom model and traditional model had almost similar positive effects on the achievement” (p. 494).
Findings from the other three studies (Sayeski et al., 2015; Adnan, 2017; Almodaires et al., 2019) were mixed. Adnan (2017)'s statistical results revealed no significant difference between the EG and the CG on midterm/quizzes and final e-portfolio scores; however, the EG achieved significantly higher essay scores. Almodaires et al. (2019) found a significant difference among the three exam results in favor of the EG but no significant difference in the final grade. Sayeski et al. (2015) found that students' learning outcomes were statistically significantly better in the flipped instructional condition than in the other two instructional conditions (homework and instructor facilitated) for a Peer-Assisted Learning Strategies (PALS) module (PALS is a reading strategy for Grades 2–6). However, there were no statistically significant differences for the Classroom Management and Accommodations modules.
Different from the abovementioned studies that examined academic performance through scores and explored the impact of FCs on students' learning outcomes, Montgomery et al. (2019) investigated the relationship between log file data and students' learning outcomes. They discovered that among all the log file data, “access day-of-the-week and access frequency” were “the strongest predictors for student success” (p. 114), while location of access had “a weak relationship” (p. 121) with academic performance.
Other Aspects of a Flipped Classroom Teaching Approach
Other aspects of FCs researched in the reviewed studies include teacher educator's perceptions (two studies), students' SLR (two studies), students' anxiety (two studies), students' self-efficacy beliefs (two studies), and students' self-perceptions of pedagogical knowledge, technological knowledge, and TPACK (one study).
Teacher Educator's Perceptions
Compared to 27 of 33 studies concerning student teachers' perceptions, there were fewer studies on teacher educators' perceptions. Only two studies, those by Ford (2015) and Tomas et al., 2019), investigated teacher educators' perceptions. The key finding of the latter study was that “additional teacher-led instruction, scaffolding and guidance were required in-class to review the concepts explored in the flipped videos, and to support students to complete the active learning tasks successfully” (p. 13). Ford (2015) shared her own experiences of teaching using the FC approach and her strategies for creating a flipped course, concluding that her experience strengthened her desire “to continue using this teaching model” (p. 378).
Students' Self-Regulated Learning
Different from Lee and Bonk (2019)'s study that investigated students' perceptions of their own SRL, Montgomery et al. (2019) utilized learning analytics (LA) to investigate students' use of SRL. They used pretests and posttests, group and individual tasks, and self-reported steps to examine whether the FC represented good pedagogy with reference to self-regulation principles. Montgomery et al. (2019)'s quantitative data about students' SRL were positive. Students were self-regulating and learning on their own by watching online lectures; students were able to acquire both content and procedure knowledge on their own from the online videos; students were able to apply their acquired online knowledge in group projects; individual students could apply the knowledge they acquired online in their own work; and students developed their self-regulation. These findings correspond with van Wyk (2018b)'s conclusion that FC pedagogy “indeed enhanced students' self-directed learning to support their learning significantly” (p. 13). According to Montgomery et al. (2019), “all six SRL behaviors [online access location, day-of-the-week, time-of-day, online frequency, online regularity and exam review patterns, our interpretation] were revealed to have weak to moderate significant relationships with academic achievement” (p. 114).
Dove and Dove (2017a,b) were interested in students' anxiety about both learning and teaching mathematics. The Dove and Dove (2017a) found “significantly greater decreases for general mathematics anxiety” (p. 134) for the EG but no significant difference in anxiety about teaching mathematics between the EG and the CG. Dove and Dove (2017b) found that the FC with teacher videos was significantly better at decreasing students' general math anxiety than the FC with lecture videos from Khan Academy and TC. It was also significantly better at decreasing students' anxiety about teaching mathematics than the FC with lecture videos from Khan Academy.
Students' Self-Efficacy Beliefs
Kurt (2017) found that students taught in FCs had “a higher level of self-efficacy beliefs” (p. 211) and that, with regard to teachers' sense of students' self-efficacy, “there was a significant difference between the experimental and control groups” (p. 215) in favor of the EG. Based on observed quantitative data, González-Gómez et al. (2019) concluded that the FC had “a significant positive impact in the science self-efficacy beliefs and attitudes toward science” (p. 9).
Students' Self-Perceptions of Pedagogical, Technological, and Technological Pedagogical Content Knowledge
Hall (2018)'s quantitative study revealed that after an FC intervention, students' self-perceptions of pedagogical knowledge and the application of TPACK were statistically significant. However, students' self-perceptions of technological knowledge and TPACK were not statistically significant.
Conclusion, Study Limitations, and Future Research
This scoping review examined 33 peer-reviewed studies on FCs in teacher education in terms of general characteristics, research methods, and research foci in the existing literature. The main findings from this review study revealed answers to aforementioned research questions. The trends in FC in teacher education are that there was a gradual and steady increase in publications on the use of FCs, most existing studies in English were conducted in the USA, Turkey, and Spain, and courses with FC within the disciplines of pedagogy and language were commonly researched. Additionally, mixed methods were more commonly used than a single quantitative or qualitative method, and surveys, test scores, and interviews were popular data collection instruments among researchers. With respect to the second research question, two main research foci were identified that researchers were primarily concerned with students' perceptions and their academic performance. The analysis of students' perceptions revealed six outcomes (1. Attitude, motivation, and emotion; 2. Content delivery; 3. Learning environment; 4. Learning experience; 5. Instructor and student presence; 6. Engagement). With regard to students' academic performance, though researchers reported dissimilar findings, more than half of their studies revealed that FC could improve students' academic performance. This research finding echoes the previous research “that non-traditional instruction are able to promote academic achievement” (Jeong et al., 2019b).
The scoping review provided an overview of FC studies conducted in the domain of teacher education. The categories developed in the review might be adapted to guide future reviews of FC pedagogy both inside and outside teacher education. Furthermore, the study findings might indicate useful directions for future FC studies to identify critical knowledge gaps and opportunities. For example, only one article (Dove and Dove, 2017a) examined consecutive FCs, and as students might need time to get used to learning with a new teaching approach, more research on consecutive FCs should be conducted.
This review study should be viewed as a pioneering attempt to explore studies on FCs in teacher education rather than an exhaustive review. Even though only 33 articles were found by searching the two databases and through a manual search, there might be other potential articles that could be found in other databases, such as Scopus and Science Direct. As FC pedagogy continues to grow in popularity, more review studies will likely be conducted and listed in more databases. Furthermore, this review utilized specific inclusion and exclusion criteria to screen identified articles. These criteria allowed the authors to narrow the scope of the search to select the most representative studies. However, different search criteria might have produced slightly different search results. For example, if book chapters were included, the data analysis might have been different. Further research should address review research identified by searching more databases and should extend to include more publication types, such as book chapters and conference proceedings.
Data Availability Statement
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.
HH is the first author of the article and has done the database searchers, analyzed the data, and written the main parts of the manuscript. FR is the second author of the article and has developed the conceptual framework, developed the research design, aided in the analysis, and writing/revisions of the manuscript.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2020.601593/full#supplementary-material
Abeysekera, L., and Dawson, P. (2015). Motivation and cognitive load in the flipped classroom: definition, rationale and a call for research. High. Educ. Res. Dev. 34, 1–14. doi: 10.1080/07294360.2014.934336
*Adnan, M. (2017). Perceptions of senior-year ELT students for flipped classroom: a materials development course. Comput. Assist. Lang. Learn. 30, 204–222. doi: 10.1080/09588221.2017.1301958
*Almodaires, A. A., Alayyar, G. M., Almsaud, T. O., and Almutairi, F. M. (2019). The effectiveness of flipped learning: a quasi-experimental study of the perceptions of kuwaiti pre-service teachers. Int. Educ. Stud. 12:10. doi: 10.5539/ies.v12n1p10
Baker, J. W. (2000). “The “classroom flip”: using web course management tools to become the guide by the side,” in Paper presented at The 11th International Conference on College Teaching and Learning. Jacksonville, FL: Florida Community College at Jacksonville.
Betihavas, V., Bridgman, H., Kornhaber, R., and Cross, M. (2016). The evidence for 'flipping out': a systematic review of the flipped classroom in nursing education. Nurse Educ. Today 38, 15–21. doi: 10.1016/j.nedt.2015.12.010
Bishop, J. L., and Verleger, M. A. (2013). “The flipped classroom: a survey of the research,” in 120th American Society for Engineering Education Annual Conference and Exposition, 30, 1–18. doi: 10.18260/1-2–22585
*Cabi, E. (2018). The impact of the flipped classroom model on students' academic achievement. Int. Rev. Res. Open Dis. Learn. 19, 202–221. doi: 10.19173/irrodl.v19i3.3482
Chapman, A. L., Morgan, L. C., and Gartlehner, G. (2010). Semi-automating the manual literature search for systematic reviews increases efficiency. Health Inform. Libr. J. 27, 22–27. doi: 10.1111/j.1471-1842.2009.00865.x
*Choi, J., and Lee, Y. (2015). To what extent does 'Flipping' make lessons effective in a multimedia production class? Innov. Educ. Teach. Int. 55, 3–12. doi: 10.1080/14703297.2015.1123105
*Conner, N. W., Rubenstein, E. D., DiBenedetto, C. A., Stripling, C. T., Roberts, T. G., and Stedman, N. L. P. (2014a). Examining student perceptions of flipping an agricultural teaching methods course. J. Agric. Educ. 55, 65–77. doi: 10.5032/jae.2014.05065
*Conner, N. W., Rubenstein, E. D., DiBenedetto, C. A., Stripling, C. T., Roberts, T. G., and Stedman, N. L. P. (2014b). Flipping an agricultural education teaching methods course. J. Agricult. Educ. 55, 66–78. doi: 10.5032/jae.2014.02066
Daudt, H. M., van Mossel, C., and Scott, S. J. (2013). Enhancing the scoping study methodology: a large, inter-professional team's experience with Arksey and O'Malley's framework. BMC Med. Res. Methodol. 13:48. doi: 10.1186/1471-2288-13-48
*Dove, A., and Dove, E. (2017a). How flipping much? Consecutive flipped mathematics courses and their influence on students' anxieties and perceptions of learning. J. Comput. Math. Sci. Teach, 36, 129–141.
*Dove, A., and Dove, E. (2017b). Flipping preservice elementary teachers' mathematics anxieties. Contemporary Issues Technol. Teach. Educ. 17, 312–335.
*Erdogan, E., and Akbaba, B. (2018). Should we flip the social studies classrooms? The opinions of social studies teacher candidates on flipped classroom. J. Educ. Learn. 7, 116–124. doi: 10.5539/jel.v7n1p116
*Ford, P. (2015). Flipping a math content course for pre-service elementary school teachers. Primus 25, 369–380. doi: 10.1080/10511970.2014.981902
*Fraga, L. M., and Harmon, J. (2014). The flipped classroom model of learning in higher education: an investigation of preservice teachers' perspectives and achievement. J. Dig. Learn. Teach. Educ. 31, 18–27. doi: 10.1080/21532974.2014.967420
*García-Sánchez, S., and Santos-Espino, J. M. (2017). Empowering pre-service teachers to produce ubiquitous flipped classes. Profile Issues in Teachers Prof. Dev. 19, 169–185. doi: 10.15446/profile.v19n1.53857
*González-Gómez, D., Jeong, J. S., Airado Rodríguez, D. A., and Cañada-Cañada, F. (2016). Performance and perception in the flipped learning model: an initial approach to evaluate the effectiveness of a new teaching methodology in a general science classroom. J. Sci. Educ. Technol. 25, 450–459. doi: 10.1007/s10956-016-9605-9
*González-Gómez, D., Jeong, J. S., and Cañada-Cañada, F. (2019). Enhancing science self-efficacy and attitudes of pre-service teachers (PST) through a flipped classroom learning environment. Interact. Learn. Environ. 1-12. doi: 10.1080/10494820.2019.1696843
*Graziano, K. J. (2017). Peer teaching in a flipped teacher education classroom. TechTrends 61, 121–129. doi: 10.1007/s11528-016-0077-9
*Hall, J. A. (2018). Flipping with the first principles of instruction: an examination of preservice teachers' technology integration development. J. Dig. Learn. Teach. Educ. 34, 201–218. doi: 10.1080/21532974.2018.1494520
*Helgevold, N., and Moen, V. (2015). The use of flipped classrooms to stimulate students' participation in an academic course in initial teacher education. Nordic J. Dig. Literacy 9, 29–42
*Jeong, J. S., González-Gómez, D., and Cañada-Cañada, F. (2019a). How does a flipped classroom course affect the affective domain toward science course? Inter. Learn. Environ. 1-13. doi: 10.1080/10494820.2019.1636079
*Jeong, J. S., Cañada-Cañada, F., and González-Gómez, D. (2018). The study of flipped-classroom for pre-service science teachers. Educ. Sci. 8:163. doi: 10.3390/educsci8040163
*Jeong, J. S., González-Gómez, D., and Cañada-Cañada, F. (2016). Students' perceptions and emotions toward learning in a flipped general science classroom. J. Sci. Educ. Technol. 25, 747–758. doi: 10.1007/s10956-016-9630-8
Jeong, J. S., González-Gómez, D., Cañada-Cañada, F., Gallego-Picó, A., and Bravo, J. C. (2019b). Effects of active learning methodologies on the students' emotions, self-efficacy beliefs and learning outcomes in a science distance learning course. J. Technol. Sci. Educ. 9, 217–227. doi: 10.3926/jotse.530
*Karaaslan, H., and Çelebi, H. (2017). ELT teacher education flipped classroom: an analysis of task challenge and student teachers' views and expectations. J. Language Ling. Stud. 13, 643–666.
*Kurt, G. (2017). Implementing the flipped classroom in teacher education: evidence from Turkey. Educ. Technol. Soc. 20, 211–221
*Lee, J., and Bonk, C. J. (2019). Integrating flipped learning with team-based learning in a pre-service teacher education course: experiences and outcomes. Int. J. eLearn. 18, 5–29
Lo, C. K., Hew, K. F., and Chen, G. (2017). Toward a set of design principles for mathematics flipped classrooms: a synthesis of research in mathematics education. Educ. Res. Rev. 22, 50–73. doi: 10.1016/j.edurev.2017.08.002
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., and the PRISMA Group (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann. Intern. Med. 151, 264–269. doi: 10.7326/0003-4819-151-4-200908180-00135
*Montgomery, A. P., Mousavi, A., Carbonaro, M., Hayward, D. V., and Dunn, W. (2019). Using learning analytics to explore self-regulated learning in flipped blended learning music teacher education. Br. J. Educ. Technol. 50, 114–127. doi: 10.1111/bjet.12590
*Ng, E. M. W. (2018). Integrating self-regulation principles with flipped classroom pedagogy for first year university students. Comput. Educ. 126, 65–74. doi: 10.1016/j.compedu.2018.07.002
O'Flaherty, J., Phillips, C., Karanicolas, S., Snelling, C., and Winning, T. (2015). The use of flipped classrooms in higher education: a scoping review. Int. High. Educ. 25, 85–95. doi: 10.1016/j.iheduc.2015.02.002
*Sammel, A., Townend, G., and Kanasa, H. (2018). Hidden expectations behind the promise of the flipped classroom. Coll. Teach. 66, 49–59. doi: 10.1080/87567555.2016.1189392
*Sayeski, K. L., Hamilton-Jones, B., and Oh, S. (2015). The efficacy of IRIS “STAR Legacy” modules under different instructional conditions. Teach. Educ. Special Educ. 38, 291–305. doi: 10.1177/0888406415600770
*Sengel, E. (2016). Using the 'flipped classroom' to enhance physics achievement of the prospective teacher impact of flipped classroom model on physics course. Comput. Assist. Lang. Learn. 30, 1–21. doi: 10.1080/09588221.2015.1111910
*Tomas, L., Evans, N., Doyle, T., and Skamp, K. (2019). Are first year students ready for a flipped classroom? A case for a flipped learning continuum. Int. J. Educ. Technol. High. Educ. 16, 1–22. doi: 10.1186/s41239-019-0135-4
*Turan, Z., and Goktas, Y. (2018). Innovative redesign of teacher education ICT courses: how flipped classrooms impact motivation? J. Educ. Future 13, 133–144.
*van Wyk, M. M. (2018a). Economics student teachers' views on the usefulness of a flipped classroom pedagogical approach for an open distance eLearning environment. Int. J. Inform. Learn. Technol. 35, 255–265. doi: 10.1108/IJILT-07-2017-0068
*van Wyk, M. M. (2018b). Flipping the economics class in a teacher education course. Technol. Knowl. Learn. 24, 373–399. doi: 10.1007/s10758-018-9377-9
Wagner, D., Laforge, P., and Cripps, D. (2013). “Lecture material retention: a first trial report on flipped classroom strategies in electronic systems engineering at the University of Regina,” in Proceedings of the Canadian Engineering Education Association Conference- École Polytechnique de Montréal. (Quebec). doi: 10.24908/pceea.v0i0.4804
*Yough, M., Merzdorf, H. E., Fedesco, H. N., and Cho, H. J. (2017). Flipping the classroom in teacher education: implications for motivation and learning. J. Teach. Educ. 70, 1–13. doi: 10.1177/0022487117742885
*^Asterisks indicate the articles included in the current review study.
Keywords: flipped classroom, teacher education, higher education, pre-service teacher, scoping review
Citation: Han H and Røkenes FM (2020) Flipped Classroom in Teacher Education: A Scoping Review. Front. Educ. 5:601593. doi: 10.3389/feduc.2020.601593
Received: 02 September 2020; Accepted: 12 October 2020;
Published: 12 November 2020.
Edited by:Jie Chi Yang, National Central University, Taiwan
Reviewed by:Jin Su Jeong, University of Extremadura, Spain
David Gonzalez-Gomez, University of Extremadura, Spain
Copyright © 2020 Han and Røkenes. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Fredrik Mørk Røkenes, email@example.com