The landscape of flipped classroom research: a bibliometrics analysis

Objective: The purpose of this study was to conduct a comprehensive review of existing research related to the flipped classroom and analyze it using bibliometrics. Methods: The search strategy of Topic = (“flipped classroom” or “invert classroom”) was used to construct a representative dataset of flipped classrooms in the Web of Science Core Collection (Science Citation Index Expanded) database up to November 2022. Author and country collaboration networks, hotspot distribution clustering, and historical citation networks associated with flipped classrooms were visualized by VOSviewer (Version 1.6.18) and R -Studio (Version 1.4). Results: A total of 937 literature (868 articles and 69 reviews) were eventually included. The number of flipped classroom studies has increased dramatically since 2012. Europe, North America, China, and Australia have more research on flipped classrooms and have significant collaborations with other countries. Since flipped classrooms involve different subject areas and collaboration among authors is relatively fragmented. Numerous flipped classroom studies have been published in high-impact journals. The thematic trend suggests that the pros and cons of flipped classrooms and their application in medical education are possible research directions for some time to come. Conclusion: Although the flipped classroom has a relatively short history, it has received a lot of attention and is currently enjoying a high level of overall acceptance. The flipped classroom has been further practiced in medical education, and we look forward to seeing its application in more subject areas (e.g., linguistics, mathematics, and computer science).


Introduction
Since the 21st century, information technology has been widely used in pedagogy (Rodríguez-Abad et al., 2022;Tsai et al., 2023).The flipped classroom has emerged as an overturning of the traditional teaching model, defined as a realignment of time in and out of the classroom, shifting learning decisions from the teacher to the student (McLaughlin et al., 2014).The proposed flipped classroom indicates that education is transforming from a teacher-centered to a student-centered teaching strategy (Persky and McLaughlin, 2017).This novel approach, i.e., flipped classroom, is an instructional approach where traditional teaching methods are reversed.Students learn new content outside class, typically through video lectures or other pre-recorded materials, and then use class time for interactive, collaborative, and applied learning activities (Rotellar and Cain, 2016).As flipped classrooms have progressed, innovative implementation methods have emerged, such as personalized learning pathways (Narang et al., 2018), peer collaboration, social learning (Xiao et al., 2018), and integration of technology (Jong, 2023), which have increased effectiveness of flipped classrooms and met a various educational need.
With the COVID-19 outbreak in 2020, the idea of "classes suspended but learning continues" is rapidly driving the development of flipped classrooms in the context of various regional containment (Durfee et al., 2020;Wang et al., 2021;Feng et al., 2022).Consequently, the flipped classroom is gaining momentum, and educators from different countries continue contributing to its development and impact (Sun et al., 2022).A randomized controlled trial conducted by American scholars (Paul et al., 2023) showed that the flipped classroom significantly improves the clinical skills of medical students compared to online-only learning.A study from China (Lu et al., 2023) suggests that flipped classrooms improve students' course learning and promote higher-order ability-set acquisition; the researchers concluded that flipped classrooms allow a rationalized formative evaluation system.Wang (2017) summarized several lessons to promote flipped classrooms, including enhancing teachers' understanding of the merits of flipped classroom approach, establishing peer mentoring mechanisms, and effective teacher capacity building.
Several recent studies have evaluated the pros and cons of flipped classrooms relative to traditional teaching.Among them, Özbay and Çınar (2021), Youhasan et al. (2021), and Barranquero-Herbosa et al. (2022), conducted a systematic review of studies occurring in the nursing discipline from different perspectives on the impact of flipped classrooms on nursing education, explains that flipped classrooms can produce positive educational outcomes and improve the quality of teaching, but that more research that meets methodological quality standards is needed to consolidate the evidence.Betihavas et al. (2016) argued that students are actively engaged when teachers introduce and rationalize the purpose of flipped classroom models to them and that continuous assessment and improvement of flipped classrooms should be reinforced.Ødegaard et al. (2021) and Xu et al. (2019) used metaanalysis to report the advantages of flipped classrooms over traditional teaching regarding knowledge acquisition and skill competence.Nevertheless, A study by Al-Said et al. (2023) reflects the potential disadvantages of flipped classrooms, which include the inability to independently study the material, a large amount of material, and technical problems.
Nevertheless, most of the studies have reviewed the literature on flipped classrooms qualitatively (Chen et al., 2021;Barranquero-Herbosa et al., 2022;Naing et al., 2023) and lack an objective description of the status of flipped classrooms research.To elucidate the situation of flipped classrooms as a new educational model in different disciplines, it is necessary to understand better the overall landscape of flipped classroom-related research and future trends (e.g., applied disciplines, applicable population).Therefore, we conducted a comprehensive bibliometric analysis of the flipped classroom-related literature, describing the applications in different disciplines, country/author collaborations, classic citations, and thematic trends.

Data sources
Bibliometric data were collected using the Web of Science Core Collection (Science Citation Index Expanded) database.The database encompasses more than 20,000 peer-reviewed, high-quality scholarly journals, including open-access journals published in more than 250 medical, social science, and humanities disciplines worldwide, and is widely used for bibliometric analysis.In addition, the database provides the authors, countries, and keywords for each publication, which was necessary for this study.
We used topic-related searches to maximize the location of content relevant to our research topic.The search strategy included Topic = ("flipped classroom" or "inverted classroom").The time horizon was limited to 2012-2022 for obtaining a recent literature overview.

Eligibility criteria
Our study had no restrictions on study population, design, or language.When non-English publications were encountered, we used DeepL for translation.Since the peer review process facilitates reliable scientific communication, stimulates meaningful research questions, and ensures accurate conclusions, we restricted the publication type of the search results to articles and reviews.In addition, by browsing titles and abstracts, we excluded articles irrelevant to flipped classroom topics.

Data analysis
All downloaded documents were imported to the VOSviewer (version 1.6.15),R (version 4.2.0), and Microsoft Excel 2019.
VOSviewer (Leiden University, Leiden, Netherlands) is a scientific knowledge-mapping software tool that uses web data to construct and visually analyze relationships and map scientific knowledge, showing structure, evolution, and collaborative relationships (van Eck and Waltman, 2010).Screened flipped classroom-related studies were imported into the VOSviewer software, and their authors and popular keywords were collected and organized.After synonymization and threshold settings were implemented, collaborative relationships between high-frequency keywords and highly productive authors were mapped, and a network cluster graph was formed.The cluster graph consists of network nodes and the connecting lines between them.The nodes represent the elements being analyzed, such as authors and keywords; their sizes represent the frequency of node occurrences; the lines between the nodes represent co-occurrence or related collaborations of the nodes, and the colors represent the different clustering relationships.
Bibliometrix, an R package for R software, allows a statistical analysis of indexes of relevant scientific literature and the study and visualization of co-citation, coupling, and co-word analysis by constructing data matrices (Aria and Cuccurullo, 2017).After importing the content of flipped classroom-related studies into the Bibliometrix package in Plain text file format, the interactive menu presents the national publication numbers, historical citation relationships, and annual hot trends of flipped classroomrelated studies.

Trends in global publication
A total of 937 literature (868 articles and 69 reviews) related to flipped classrooms were retrieved.The earliest record of flipped classroom research in the Web of Science Core Collection database was in 2012, with three literatures reported.The number of articles published in this field has increased over the past ten years.In particular, the average annual number of articles published in the field was close to 50 in 2015 (n = 43) and 100 in 2018, reaching a peak of 186 in 2021.The time curve constructed by the logistic regression model suggested that the field is currently in a phase of steady growth in global publication output (Figure 1).

Distribution of countries/regions
The results show that flipped classroom studies have been conducted in 73 countries.The top five countries with the highest number of scientific results are the United States (363), China (207), Spain (80), Germany (44), and Australia (42).The United States was the first to publish flipped classroom research results, and since 2021, the number of Chinese publications has proliferated, surpassing that of the United States (Figure 3A).In addition to the number of scientific results, a country's research strength can be measured by national collaborations.Figure 3B depicts global partnerships, with the United States having the most national links, followed by China, Australia, the United Kingdom, and Canada.Other countries have fewer collaborations in flipped classroom research, with fewer than 15 connections.

Most influential authors
The h-index is based on the number of times a particular scientist's paper is cited and is a widely accepted measure of scientific performance.The top-ten authors of h-index are shown in Table 1.
Mclaughlin JE is the most influential researcher and the earliest recorded author of flipped classroom research in the Web of Science database.We note that the high-impact ten authors are primarily from the United States, Spain, and China, indicating that education scholars Cluster collaboration analysis was conducted using VOSviewer for authors of three or more articles, and 56 authors were grouped into 20 clusters (Supplementary Figure S1).The collaborative network mapping among authors showed a "blossoming" pattern, which indicates that flipped classroom research is being adopted and applied in multiple disciplines.

Most influential source journals
Flipped classroom research has appeared in 258 journals, with the top five publishing 220 papers (23.5%), while 158 journals (61.2%) have published only one paper on the flipped classroom.Twenty-two journals (8.5%) published more than ten papers.As shown in Figure 4, the top five journals in terms of several published articles were BMC  2).
The source journals for flipped classrooms research papers were highly dispersed according to Bradford's law (Supplementary Figure S2); the top ten most influential journals were selected based on the number of local citations (Table 2).According to Bradford's law, the journals marked with an asterisk are the core source journals in flipped classroom research, including Computers & Education, Journal of Chemical Education, American Journal of Pharmaceutical Education, BMC Medical Education, and Advances in Physiology Education.Thus, these journals played a vital role in flipped classroom research during the study period.

Study of hotspots and co-occurring keyword clustering network analysis
We generated word clouds through the hotspot analysis function of the biblimetrix package that can reveal the most famous content in the field (Supplementary Figure S3).Active learning, blended learning, and medical education have become the most popular topics in the flipped classroom series of studies.The 39 author's keywords with a frequency of 10 or more were analyzed, and VOSviewer divided them into seven clusters and formed a network connection, revealing the main areas and directions of development (Figure 5).Different colors distinguish different clusters, representing a category of research hot directions.Cluster 1 is the largest group, including 12 keywords, mainly related to active learning,

Citation classics
Table 3 shows the top-ten most cited flipped classroom-related studies."The Flipped Classroom: A Course Redesign to Foster Learning and Engagement in a Health Professions School, " published by Mclaughlin JE et al. in Academic Medicine in 2014, received the most frequent citations (total citation: 577).This study reports on the effectiveness of flipped classrooms compared to traditional instruction on student application in the pharmaceutics course, describing the advantages of flipped classrooms on class attendance, students' learning, and perceived value.

Trends in flipped classroom research and themes distribution
Figure 6 shows the relationship of flipped classroom-related research hotspots over time, which can reflect the trending topics of flipped classroom research.The span of the horizontal line indicates the period of word explosion, and the circle size refers to the word frequency.Undergraduate medical education, COVID-19, and nursing dominate the top three positions in flipped classroom-related research.They will likely continue to be popular research topics as the current global outbreak is still developing.The call for educational reform is driving the development of new teaching models such as flipped classrooms, blended learning, active learning, and student-centered learning, which can also be found in the word cloud and will continue to be followed in the future.
Supplementary Figure S4 shows the distribution and future trends of current research topics related to flipped classrooms in the form of coordinates.The first to fourth quadrants represent motor, niche, emerging, declining, and basic themes.Motor themes include flipped classrooms, active learning, and blended learning, all of which have significant trends.Curriculum reform and distance learning are the cornerstones of flipped classroom research.They need to be further explored while contributing to developing flipped classroomrelated research.

Discussion
The main findings of this study The "flipped classroom" is a complete reversal of the traditional classroom teaching structure and process based on printing, which leads to a series of reforms in teachers' roles and curriculum models (Anderson et al., 2017).This study reviewed 937 flipped classroomrelated papers published in the Web of Science Core Collection database and found that research involving flipped classrooms has gradually grown since the flipped classroom was reported in 2012.The COVID-19 outbreak in late 2019 has somewhat boosted the  Temporal trends in author's keywords.(Bergmann and Sams, 2012;Consortium, 2014), we charted an evolution of flipped classroom adoption through the perspective of historical development (Figure 7).This new teaching model is gradually gaining attention and recognition among educators and students.Flipped classroom development varies greatly among countries, with Australia, China, Germany, Spain, and the United States leading the way.As information technology advances, the actual effectiveness of flipped classrooms versus traditional instruction will continue to be a primary research focus for educators for some time to come.
As a relatively new educational model, the flipped classroom has not yet received widespread attention from educators.Scholars from the United States, Spain, and China have some strengths in this area and dominate the research component, but collaboration between different groups is relatively sparse, which may be related to the conduct of different disciplines.For example, some researchers have applied flipped classrooms to medical education (Ruzafa-Martínez et al., 2023;Wade et al., 2023), and other educators have introduced the model to language subjects (Guo et al., 2016;Yin, 2017;Sa et al., 2022) and mechanical engineering (Cho et al., 2021).We look forward to furthering cross-collaboration to promote the development of flipped classroom-related research.Flipped classroom research is conducted worldwide, with some differences and imbalances remaining.Europe, North America, China, and Australia have more research on flipped classrooms and have significant collaborations with other countries.As more and more scholars worldwide gradually recognize the feasibility and significance of the flipped classroom, it is bound to progress and develop soon.Although the concept of flipped classrooms is relatively new, its related research is still widely accepted, and most of the flipped classroom studies are published in high-quality journals, such as Computers & Education, Academic Medicine, and Medical Education.
In 2011, the book "Flip Your Classroom: Reach Every Student in Every Class Every Day" was published, which officially opened the door to flipped classroom research (Bergmann and Sams, 2012).In the same year, the concept of flipped classrooms was introduced to China, and "flipped classrooms" became a buzzword in education (McDonald and Smith, 2013;Sharma et al., 2015).Nevertheless, the research on flipped classrooms is still relatively small, and the field is still at an early stage of development, with great development space.Throughout the history of flipped classrooms, we can find that as a new teaching model, there are particular controversies, such as students' initiatives directly affecting the effectiveness of flipped classrooms (Butt, 2014;Kang and Kim, 2021); Then, for example, the use and management of technology (Moraros et al., 2015).This has also given rise to active learning facilitation strategies.On the other hand, the global popularity of COVID-19 has led to massive online education, which poses higher requirements for implementing the flipped classroom and shows the potential of flipped classroom development to some extent (Lo and Hew, 2022).
The research hotspots on flipped classrooms are relatively concentrated.The focus is on the advantages of flipped classrooms over traditional teaching (e.g., personalized learning, developing critical thinking, self-directed learning, and improving teacher effectiveness), application of flipped classrooms in different disciplines, especially medical education, and the exploration of other learning models based on flipped classrooms.After the popularity of flipped classrooms, educators have been exploring how to help students dramatically improve their performance (Rotellar and Cain, 2016).The earliest flipped classrooms were taught online through instructional videos, and this model is popular today (Oudbier et al., 2022).Later, flipped classrooms began to be integrated into the offline learning phase, an initiative to address curriculum problems encountered in online learning that could be solved at school.With the rapid development of information technology, the flipped classroom gradually entered the flipped technology phase, which means that more emphasis is placed on new approaches (e.g., audio, games) or new technologies to facilitate teacherstudent interaction (Gong and Zhou, 2022;Wu et al., 2022).On the other hand, various empirical studies have proved the feasibility and necessity of flipped classrooms, which further promotes the practical approach of flipped classrooms and provides help and reference for future applications in more disciplines (Gray et al., 2022;Wen et al., 2022;Paul et al., 2023).
Healthcare providers are an essential part of the healthcare system.The world is still experiencing a severe epidemic caused by COVID-19 (Worldometer, 2022), and healthcare providers are a necessary element in the treatment process.To fulfill their role,  (Khodaei et al., 2022), dentistry (Özcan, 2022), and pharmaceutical education (Peng et al., 2022).It is foreseeable that the flipped classroom-based teaching model will play an important role in medical education in the future (Divjak et al., 2022).

Comparison with similar studies
The current bibliometric results are like the study by Kushairi and Ahmi (2021); both indicate that flipped classrooms will gain increasing attention from practitioners, researchers, and educational scholars in the coming years.The difference is that the latter used the Scopus database to access potential literature and Lotka's Law to describe the relationship between literature and number of authors.Our study is refined to a disciplinary distribution, reporting on flipped classrooms' practicability and sustaining characteristics for different pedagogical domains.Furthermore, we mapped the keyword co-occurrence network and elaborated the research hotspots, providing pathways for future flipped classroom research.
In addition, we recognize that the current study is like another one (del Arco et al., 2022) in that it focuses on Web of Science research areas, country/region publications, published journals, and keywords, but we also present dynamic graphs, which provide more feedback on the trends in flipped classroom research, as shown in Figures 2B, 3A, 4, 6.Not only that, but we also propose several research hotspots and future study directions based on the keyword co-occurrence analysis.Interestingly, several econometric analyses on flipped classrooms illustrate the considerable research interest in this area.

Contribution of this study and future directions
The results of this study will help educational researchers understand the success of flipped classroom models worldwide and suggest avenues for further research.In addition, this study shows that significant progress has been made in teaching models based on flipped classrooms, especially in medical education.However, we can also foresee that flipped classrooms will continue to expand to other disciplines, including linguistics, mathematics, and computer science.As the flipped classroom integrates across multiple disciplines, there will be closer collaboration between researchers and even countries in the future.For the following research direction, we propose several directions.First, how to better integrate and optimize modern educational technologies, such as artificial intelligence, virtual reality, and augmented reality, to enhance the effect and interactive experience of the flipped classroom.Second, exploring the design of personalized learning paths based on student's learning styles, interests, and abilities to meet the learning needs of different students and improve learning effectiveness.Third, develop assessment and measurement tools applicable to the flipped classroom to measure the development of students' academic achievement, independent learning ability, and teamwork ability.Fourth, provide teachers with relevant training, resources, and support to help them better apply the flipped classroom teaching model.

Limitations
Although this article analyzes the research landscape of flipped classrooms from different perspectives, there are some unavoidable limitations.First, we only searched the Web of Science Core Collection database, and there are biases in the quantification and visualization of the literature, especially the citation classics, and possibly other influential articles were neglected.Second, since only two search terms for the flipped classroom were used, we may have missed flipped classroom-related articles because Jon Bergmann and Aaron Sams pioneered the flipped classroom in 2007, and there were initial attempts, whereas the earliest literature we retrieved in Web of Science was published in 2012.Third, bibliometric analysis is only an auxiliary tool, and the results may differ from real-world research conditions.Fourth, because citations change over time, bibliometric data may yield different conclusions at different points in time.

Conclusion
This review provides a comprehensive overview of the flipped classroom research field from 2012 to 2022 using bibliometric analysis.

The
Web of Science research areas assigned by Clarivate Analytics is used to classify research papers.Each paper can be organized into at least one Web of Science database research area.In this study, the number of research areas covered by the flipped classroom literature increased from 4 in 2012 to 44 in 2021 (Figure 2A).The top ten most productive research areas are chemistry, computer science, dentistry oral surgery & medicine, education & educational research, engineering, environmental sciences & ecology, health care sciences & services, nursing, telecommunications, and science & technology-other topics, representing 720 out of 868 publications, or approximately 78.9% of the total, Figure 2B illustrates the annual evolution of the ten areas with the most flipped classroom research results.Standing out in Figure 2B is education & educational research, reflecting the increasing research on teaching and learning in the flipped classroom over time, suggesting that this new teaching model has become the development direction of international teaching and has gradually been recognized by education researchers.

FIGURE 1
FIGURE 1Global trends in publications on flipped classrooms research.
FIGURE 2 (A) Number of Web of Science research areas covered in flipped classrooms-related literature.(B) Temporal evolution of the top-ten most productive Web of Science research areas in flipped classrooms-related literature.
FIGURE 3 (A) Top five countries according to annual scientific production.(B) Map showing research collaboration between countries.

FIGURE 4
FIGURE 4Temporal analysis of the publication source of flipped classrooms-related research.

FIGURE 5
FIGURE 5Clustering distribution of keywords co-occurrence hotspot network.

FIGURE 6
FIGURE 6 development of flipped classrooms.Therefore, we take 2020 as the boundary, and 423 (45.14%) flipped classroom-related articles were identified by Web of Science before the outbreak, while 514 (54.86%) articles were published in just three years from 2020 to 2022.Following the reports in the literature FIGURE 7Historical development of flipped classroom applications.
Regarding the global trends in the scientific literature on flipped classrooms, the number of publications has increased exponentially over the past decades, covering a more comprehensive range of research areas.The United States, China, Spain, Germany, and Australia are the major research countries; BMC Medical Education is the more authoritative journal publishing flipped classroom studies; Mclaughlin JE is the most prolific and influential author.The topic trends identified in this study indicate that flipped classroom studies data sources are becoming more abundant and cover a broader range of research areas.The focus is on the advantages of flipped classrooms over traditional teaching, the application of flipped classrooms in different disciplines, especially medical education, and the exploration of other learning models based on flipped classrooms.Future flipped classroom research could center on innovative use of technology tools and platforms, learning motivation and engagement, long-term learning effectiveness and knowledge transfer, and sustainability and implementation strategies.

TABLE 2
The top-ten journals in flipped classrooms-related field.
*The journal is the core resource (classified by Bradford Law) of flipped classrooms research.

TABLE 1
The distribution of the top-ten authors with institutions and countries.

TABLE 3
Top-ten cited flipped classroom-related studies.