A review of integrating AI-based chatbots into flipped learning: new possibilities and challenges

Introduction

Artificial intelligence (AI) conversation chatbots have gained significant attention worldwide, especially after the release of ChatGPT by OpenAI¹ on November 30, 2022. Although ChatGPT has created an impact on different disciplines, it is widely reported that it relies on biased data and may provide incorrect or fake information (Lo, 2023). Therefore, there is still a need to build chatbots for specific purposes, such as guiding student learning in a course. An AI-based chatbot is a computer program designed to simulate human conversation through natural language processing to understand and respond to user queries in a human-like manner. These chatbots are thus valuable tools in various industries, from customer service (Nicolescu and Tudorache, 2022) to healthcare (Xu et al., 2021), by improving workflow efficiencies, reducing costs, and enhancing user experience.

In the education sector, Wollny et al. (2021) have classified AI chatbots into three major types: assisting chatbots (e.g., course assistance), mentoring chatbots (e.g., increasing self-awareness), and learning chatbots which are specifically trained to support students’ learning of course materials. For example, Lee and Yeo (2022) developed an AI-based chatbot to act as a virtual student who had difficulty learning mathematics. This chatbot was designed to enhance pre-service teachers’ questioning skills through a 24/7 simulation of student-teacher dialogue.

There is a great potential to enhance flipped learning with AI chatbots (Diwanji et al., 2018). Flipped learning is an instructional approach that reverses the traditional order of class activities (Bishop and Verleger, 2013). In a typical flipped lesson, students engage with learning materials (e.g., instructional videos and readings) before class and then come to class prepared with basic knowledge to participate in more advanced and interactive learning activities (Hew et al., 2021a; Lo and Hew, 2022). Although recent meta-analyses have suggested that flipped learning could increase student achievement across subject disciplines (Cheng et al., 2019; van Alten et al., 2019; Hew et al., 2021a), its practices are not without challenges. As Akçayır and Akçayır (2018) found, two common complaints of flipped learning are that students lack guidelines at home, and that they are unable to get help during pre-class learning, which impedes their engagement in subsequent in-class activities. Ideally, AI chatbots can be used to provide them with 24/7 assistance and personalized support (Diwanji et al., 2018).

However, integrating AI chatbots into flipped learning is a new and emerging concept. While there have been studies on the use of AI chatbots in education (see Wollny et al., 2021 for a review) and flipped learning (see Hew et al., 2021a for a review) separately, existing reviews have found a scarcity of studies on their integration. In a recent systematic review by Wollny et al. (2021) on chatbots in education, only one study of flipped learning (i.e., Huang et al., 2019) was retrieved as of December 23, 2020. This indicates that it is still a relatively new research area worthy of a follow-up literature review, as the new development of AI chatbot technology may have led to more effective ways of using them in flipped learning and new challenges that need to be addressed. Hence, this mini review aims to understand recent practices for designing and integrating AI chatbots in flipped learning, their benefits, and challenges. Our findings can serve as a foundation for further research and development in this area. The following research questions (RQ1 to RQ3) are posed to guide this review.

• RQ1: How are AI chatbots designed and used in flipped learning?

• RQ2: What are the benefits of integrating AI chatbots into flipped learning?

• RQ3: What are the challenges of integrating AI chatbots into flipped learning?

Methods

Search strategies

We followed the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) statement (Moher et al., 2009) when selecting relevant articles. Eleven electronic databases were used, including (1) Academic Search Ultimate, (2) ACM Digital Library, (3) APA PsycInfo, (4) British Education Index, (5) CINAHL, (6) Education Research Complete, (7) ERIC, (8) IEEE Xplore, (9) PubMed, (10) Scopus, and (11) Web of Science. The search string was designed based on recent literature reviews of AI chatbots (Wollny et al., 2021; Nicolescu and Tudorache, 2022) and flipped learning (Hew et al., 2021a; Lo and Hew, 2022), and it was formulated as follows: (“conversational AI” OR “dialogue system*” OR “dialog system*” OR “chatbot*” OR “conversational agent*”) AND (flip* OR invert*) AND (class* OR learn* OR instruction* OR course*). The Boolean operators and asterisks were used to increase the flexibility of the search string in retrieving relevant articles of AI chatbots and flipped learning. A snowballing procedure was further executed using Google Scholar. Specifically, we tracked the research items which cited the included articles.

Inclusion and exclusion criteria

We conducted our final search on 30 April 2023 without specifying a period of publication. Therefore, relevant articles published before the search date were identified and screened. To answer our research questions, only empirical studies were included. However, no constraints were imposed on the types of empirical data used (e.g., surveys and interviews). To ensure consistency, the interventions had to satisfy the definition of flipped learning (i.e., students first engage with learning materials before class and then participate in in-class activities; Cheng et al., 2019; Lo and Hew, 2022) and use at least one chatbot in their practice. Considering the integration of AI chatbots and flipped learning is an emerging research area, we included conference papers and book chapters in addition to journal articles. No constraints were imposed on the location of interventions, research contexts, and language of instruction. However, the articles must be written in English. Table 1 summarizes the inclusion and exclusion criteria for study selection.

TABLE 1

Table 1. Inclusion and exclusion criteria for study selection.

Data extraction and analysis

We extracted data from each article, including the authors(s), the year of publication, the country of implementation, the subject area, and the research participants. We further extracted information on how AI chatbots were designed and used in flipped learning (RQ1), how the integration benefited teachers and students (RQ2), and the challenges to the integration (RQ3). The data were analyzed through content analysis (Creswell, 2012), and emerging themes were identified through open coding. Similar codes were grouped and organized into themes. Exemplary quotations were identified to illustrate each constructed theme. The articles were double-coded to establish coding reliability, and disagreements were resolved through discussion among the authors.

Findings and discussion

Study selection and characteristics of the included articles

A total of 31 records were obtained through a database search as of 30 April 2023. After removing duplicates, there were 16 unique records. However, some were excluded because they were not related to flipped learning (N = 5), not written in English (N = 2), nor empirical studies (N = 1). One study was excluded after full-text assessment for eligibility due to a lack of empirical data. Nevertheless, it was used for background reference. Hence, seven articles were yielded. We then tracked the research items which cited the included articles using Google Scholar. After executing this snowballing procedure, three additional articles were included. Overall, the final selection yielded 10 articles for synthesis. Notably, Du et al. (2021) stated that their study was a follow-up study of Hew et al. (2021b). Therefore, a total of nine unique studies were analyzed. Figure 1 outlines the process of article selection.

FIGURE 1

Figure 1. PRISMA flow diagram of article selection procedure.

The background of the reviewed studies is summarized in the Appendix. A majority of studies were conducted in Asia, such as Hong Kong (Gonda and Chu, 2019; Huang et al., 2019; Hew et al., 2021b 2023; Li et al., 2021), Japan (Ito et al., 2021), and Taiwan (Lin and Mubarok, 2021). With the exception of the study by Timpe-Laughlin et al. (2022), all other studies involved students in higher education. However, the reviewed studies were conducted in various subject areas. Only the studies by Lin and Mubarok (2021), Timpe-Laughlin et al. (2022), and Hew et al. (2023) were within the same subject area (i.e., English language education).

RQ1: how are AI chatbots designed and used in flipped learning?

In the reviewed studies, only Lin and Mubarok (2021) adopted a ready-made chatbot application, called Replika, which allowed students to interact with and practice their English speaking. Researchers in other studies built their own chatbots for flipped learning. For example, Varnavsky (2022) used Python programming to develop his chatbot and incorporated it with Telegram (an instant messaging application). Li et al. (2021) detailed their system parameters and variables considered (e.g., students’ fraction of correctly answered pre-class quizzes and the time needed) when developing their chatbot. The chatbot was then implemented on Facebook Messenger and Telegram. Besides, Gonda and Chu (2019) and Hew et al. (2021b, 2023) built their chatbots using an existing platform, namely Google Dialogflow, whereas Huang et al. (2019) used another platform called IBM Watson Assistant. They had to customize the chatbots by setting intents (i.e., users’ possible questions), entities (i.e., keywords which help the chatbots recognize users’ words), and dialogue relevant to their courses.

Consistent with Wollny et al. (2021), the chatbots involved in the reviewed studies could be classified into three categories as follows (the total number is greater than nine because the chatbots in some studies served multiple purposes).

• Learning chatbot (N = 9): The chatbots in all reviewed studies were used to facilitate students’ out-of-class learning. They could provide real-time feedback on student performance (Gonda and Chu, 2019; Huang et al., 2019; Hew et al., 2021b, 2023; Li et al., 2021), answer students’ questions (Gonda and Chu, 2019; Huang et al., 2019; Ito et al., 2021; Varnavsky, 2022), and allow students to practice their English speaking (Lin and Mubarok, 2021; Timpe-Laughlin et al., 2022).

• Assisting chatbot (N = 2): Varnavsky’s (2022) chatbot could serve as a course assistant, such as “Providing material or links to material to be studied in preparation for the current workshop” (p. 291). In Li et al. (2021), “Students using Facebook Messenger chatbot in 2019 received their messages (i.e., quizzes and prompts) automatically (just like the Telegram user)” (p. 124).

• Mentoring chatbot (N = 2): We found two mentoring chatbots, namely “Self-Regulated Learning Chatbot” (Hew et al., 2021b, p. 169) and “goal-setting chatbot” (Hew et al., 2023, p. 40). Both of these helped students set their personal learning goals and then provided recommendations for them to achieve their goals.

RQ2: what are the benefits of integrating AI chatbots into flipped learning?

Several reviewed studies provided evidence that integrating chatbots into flipped learning was useful (Hew et al., 2021b, 2023) and enhanced student learning (Li et al., 2021). The following five specific benefits were identified.

• Immediate feedback (N = 4): Chatbots could provide students with timely feedback to support their learning and achievement of learning goals (Gonda and Chu, 2019; Huang et al., 2019; Hew et al., 2021b. In the words of one student, the chatbot provided prompt feedback which “solved students’ problems in time” (Hew et al., 2021b, p. 173). Ito et al. (2021) further noted that students could ask their chatbot during class and even at midnight when they did their homework.

• Increased students’ interaction with learning content (N = 4): Chatbot-integrated learning led to an increase in students’ interaction with course materials (Huang et al., 2019; Hew et al., 2021b, 2023). In their English-speaking course, Lin and Mubarok (2021) further used mind maps to guide students’ interactions with their chatbot, resulting in a significant increase in student-chatbot interaction as evidenced by the greater number of words produced.

• Improved students’ class preparation (N = 3): The use of chatbots sending reminders to students led to a significant decrease in the percentage of students who did not read pre-class materials (Varnavsky, 2022). Li et al. (2021) found that around 90% of their students completed their pre-class quizzes within 3 days with messages pushed in their chatbots. Gonda and Chu (2019) further noted that their chatbot could encourage their student to think and ask questions outside the classroom.

• Increased confidence (N = 2): This benefit was identified in the studies of English-speaking courses. Speaking with chatbots could boost students’ confidence, as perceived by the teacher participants of Timpe-Laughlin et al. (2022). Similarly, Lin and Mubarok (2021) reported that their students felt more confident in their conversations.

• Data-driven teaching and learning (N = 1): The chatbot of Li et al. (2021) collected and analyzed students’ performance in pre-class and in-class quizzes. Based on the analytics, the chatbot could provide pre-class quizzes that matched their level of proficiency and notify the teacher to modify the level of difficulty for learning tasks.

RQ3: what are the challenges of integrating AI chatbots into flipped learning?

While integrating chatbots into flipped learning can provide the aforementioned benefits, several challenges were identified in the reviewed studies, as follows.

• Limited technical functionality (N = 4): Hew et al. (2021b, 2023) pre-defined several options in their chatbot to assist students in labeling their input. However, it appeared that the variety of options was inadequate to fully meet the needs of their students. Huang et al. (2019) also noted that their chatbots could not solve unstructured problems requested by their students. In the study of Timpe-Laughlin et al. (2022), some teacher participants experienced difficulty with the chatbot’s speech recognition and response accuracy.

• Lacking authenticity (N = 2): Timpe-Laughlin et al. (2022) reported concerns about the authenticity of learning tasks when using chatbots. The limited function of their chatbot might have restricted task complexity. Echoed with Timpe-Laughlin et al. (2022), one student of Huang et al. (2019) expressed that “I only want to ask very conceptual questions (what is…) from the chatbots, and as for some more complex questions (why…, how…), I prefer to ask a human friend” (p. 817).

• Lacking students’ learning motivation (N = 2): Varnavsky (2022) reported that not all students used the chatbot throughout the course, and he attributed it to students’ interest and motivation in the subject discipline. Similarly, Ito et al. (2021) found it challenging to motivate students during online learning, even with the presence of a chatbot.

Limitations and recommendations for future research and development

Two main limitations in this review must be acknowledged. First, although multiple databases were searched with a flexible search string, only 10 relevant empirical studies could be found. While this review could provide valuable insights into the use of AI chatbots in flipped learning, the small number of reviewed studies limited the universality and generalizability of our findings. Moreover, because the reviewed studies were conducted in diverse subject areas, it is difficult to draw broad conclusions in this research topic. Hence, we recommend future studies be conducted in a range of subject areas (e.g., mathematics and health professions education) and contexts (e.g., K-12 schools) to advance our knowledge of the potential benefits and challenges of integrating AI chatbots into flipped learning.

Second, the findings on the effectiveness and usefulness of integrating AI chatbots into flipped learning were largely based on self-reported data (e.g., surveys and interviews). Only one study used achievement tests to assess students’ learning outcomes (Lin and Mubarok, 2021). Therefore, we currently know little about the effect of using chatbots in flipped learning as compared with some other learning environments (e.g., flipped learning without using a chatbot). Further experimental research is required to examine the effectiveness of chatbot-supported flipped learning.

To address the challenges identified in this review, we provide an agenda for future research and development. First, it is necessary to develop a comprehensive database of Q&A data to enhance the chatbot’s technical functionality and provide a wider range of responses to student queries (Ito et al., 2021). Involving frontline teachers in the chatbot and learning task design process can help ensure the authenticity of learning tasks and enhance the chatbot’s technical functionality (Timpe-Laughlin et al., 2022). They can provide subject-specific insights into the types of questions and tasks that are relevant to their subject areas, which can become the basis for training data. In addition, to improve student motivation and engagement, allocating scores on learning tasks might be able to serve as an incentive to encourage class preparation (Li et al., 2021; Lo and Hew, 2022). However, to enhance intrinsic motivation, Diwanji et al. (2018) recommended the use of Self-determination Theory (Ryan and Deci, 2000) in chatbot and learning task designs. To enhance student engagement with chatbots, one student of Hew et al. (2021b) suggested embedding emojis in the chatbot responses in their chatbot. This suggestion echoed with the attempt of Lee and Yeo (2022) who made the responses of their chatbot more human-like by adding emojis and emotional reactions.

Conclusion

This mini review on integrating AI chatbots into flipped learning is significant in advancing our understanding of the state-of-the-art in this topic and its potential to enhance the learning experience for students. Our review identified the benefits (e.g., improved class preparation and data-driven teaching and learning) and potential challenges (e.g., limited technical functionality and lacking authenticity), which can inform educators and instructional designers in their implementation of chatbot-supported flipped learning. Most importantly, we provided recommendations for future research and development to address the challenges identified in this review. Nevertheless, more studies are required to investigate the effectiveness of chatbot-supported flipped learning in various educational contexts.

Author contributions

Both authors listed have made a substantial, direct, and intellectual contribution to the work, and approved it for publication.

Funding

This research was funded by the Department of Mathematics and Information Technology, the Education University of Hong Kong, reference code MIT/DRG01/22-23.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^ https://openai.com

References

Akçayır, G., and Akçayır, M. (2018). The flipped classroom: A review of its advantages and challenges. Comput. Educ. 126, 334–345. doi: 10.1016/j.compedu.2018.07.021

CrossRef Full Text | Google Scholar

Bishop, J. L., and Verleger, M. A. (2013). “The flipped classroom: A survey of the research,” in Proceedings of 2013 ASEE annual conference & exposition, (Atlanta, GA: ASEE), 6219. doi: 10.18260/1-2-22585

CrossRef Full Text | Google Scholar

Cheng, L., Ritzhaupt, A. D., and Antonenko, P. (2019). Effects of the flipped classroom instructional strategy on students’ learning outcomes: A meta-analysis. Educ. Tech. Res. Dev. 67, 793–824. doi: 10.1007/s11423-018-9633-7

CrossRef Full Text | Google Scholar

Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research., 4th Edn. Boston, MA: Pearson Education, Inc.

Google Scholar

Diwanji, P., Hinkelmann, K., and Witschel, H. F. (2018). “Enhance classroom preparation for flipped classroom using AI and analytics,” in Proceedings of the 20th international conference on enterprise information systems – volume 1: ICEIS, (Funchal: Science and Technology Publications, Lda), 477–483. doi: 10.5220/0006807604770483

CrossRef Full Text | Google Scholar

Du, J., Huang, W., and Hew, K. F. (2021). “Supporting students goal setting process using chatbot implementation in a fully online course,” in Proceedings of 2021 IEEE international conference on engineering, technology & education, (Wuhan: IEEE), 35–41. doi: 10.1109/TALE52509.2021.9678564

CrossRef Full Text | Google Scholar

Gonda, D. E., and Chu, B. (2019). “Chatbot as a learning resource? Creating conversational bots as a supplement for teaching assistant training course,” in Proceedings of 2019 IEEE international conference on engineering, technology and education, (Yogyakarta: IEEE), 1–5. doi: 10.1109/TALE48000.2019.9225974

CrossRef Full Text | Google Scholar

Hew, K. F., Bai, S., Dawson, P., and Lo, C. K. (2021a). Meta-analyses of flipped classroom studies: A review of methodology. Educ. Res. Rev. 33:100393. doi: 10.1016/j.edurev.2021.100393

CrossRef Full Text | Google Scholar

Hew, K. F., Huang, W., Du, J., and Jia, C. (2021b). “Using chatbots in flipped learning online sessions: Perceived usefulness and ease of use,” in Blended learning: re-thinking and re-defining the learning process, eds R. Li, S. K. S. Cheung, C. Iwasaki, L. F. Kwok, and M. Kageto (Cham: Springer), 164–175. doi: 10.1007/978-3-030-80504-3_14

CrossRef Full Text | Google Scholar

Hew, K. F., Huang, W., Du, J., and Jia, C. (2023). Using chatbots to support student goal setting and social presence in fully online activities: Learner engagement and perceptions. J. Comput. High. Educ. 35, 40–68. doi: 10.1007/s12528-022-09338-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Huang, W., Hew, K. F., and Gonda, D. E. (2019). Designing and evaluating three chatbot-enhanced activities for a flipped graduate course. Int. J. Mech. Eng. Rob. Res. 8, 813–818. doi: 10.18178/ijmerr.8.5.813-818

CrossRef Full Text | Google Scholar

Ito, T., Tanaka, M. S., Shin, M., and Miyazaki, K. (2021). “The online PBL (project-based learning) education system using AI (artificial intelligence),” in DS 110: Proceedings of the 23rd international conference on engineering and product design education, (Herning: Design Society), doi: 10.35199/EPDE.2021.19

CrossRef Full Text | Google Scholar

Lee, D., and Yeo, S. (2022). Developing an AI-based chatbot for practicing responsive teaching in mathematics. Comput. Educ. 191:104646. doi: 10.1016/j.compedu.2022.104646

CrossRef Full Text | Google Scholar

Li, J., Ling, L., and Tan, C. W. (2021). “Blending peer instruction with just-in-time teaching: Jointly optimal task scheduling with feedback for classroom flipping,” in Proceedings of the Eighth ACM conference on learning @ scale, (Germany: ACM), 117–126. doi: 10.1145/3430895.3460134

CrossRef Full Text | Google Scholar

Lin, C. J., and Mubarok, H. (2021). Learning analytics for investigating the mind map-guided AI chatbot approach in an EFL flipped speaking classroom. Educ. Technol. Soc. 24, 16–35. doi: 10.30191/ETS.202110_24(4).0002

CrossRef Full Text | Google Scholar

Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Educ. Sci. 13:410. doi: 10.3390/educsci13040410

CrossRef Full Text | Google Scholar

Lo, C. K., and Hew, K. F. (2022). Design principles for fully online flipped learning in health professions education: A systematic review of research during the COVID-19 pandemic. BMC Med. Educ. 22:720. doi: 10.1186/s12909-022-03782-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., and The Prisma Group. (2009). Reprint-preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Phys. Ther. 89, 873–880. doi: 10.1093/ptj/89.9.873

CrossRef Full Text | Google Scholar

Nicolescu, L., and Tudorache, M. T. (2022). Human-computer interaction in customer service: The experience with AI chatbots—A systematic literature review. Electronics 11:1579. doi: 10.3390/electronics11101579

CrossRef Full Text | Google Scholar

Ryan, R. M., and Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am. Psychol. 55, 68–78. doi: 10.1037/0003-066X.55.1.68

PubMed Abstract | CrossRef Full Text | Google Scholar

Timpe-Laughlin, V., Sydorenko, T., and Daurio, P. (2022). Using spoken dialogue technology for L2 speaking practice: What do teachers think? Comput. Assist. Lang. Learn. 35, 1194–1217. doi: 10.1080/09588221.2020.1774904

CrossRef Full Text | Google Scholar

van Alten, D. C., Phielix, C., Janssen, J., and Kester, L. (2019). Effects of flipping the classroom on learning outcomes and satisfaction: A meta-analysis. Educ. Res. Rev. 28:100281. doi: 10.1016/j.edurev.2019.05.003

CrossRef Full Text | Google Scholar

Varnavsky, A. N. (2022). “Chatbot to increase the effectiveness of the «flipped classrooms» technology,” in Proceedings of 2022 2nd international conference on technology enhanced learning in higher education, (Lipetsk: IEEE), 289–293. doi: 10.1109/TELE55498.2022.9801001

CrossRef Full Text | Google Scholar

Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., and Drachsler, H. (2021). Are we there yet? A systematic literature review on chatbots in education. Front. Artif. Intell. 4:654924. doi: 10.3389/frai.2021.654924

PubMed Abstract | CrossRef Full Text | Google Scholar

Xu, L., Sanders, L., Li, K., and Chow, J. C. (2021). Chatbot for health care and oncology applications using artificial intelligence and machine learning: Systematic review. JMIR Cancer 7:e27850. doi: 10.2196/27850

PubMed Abstract | CrossRef Full Text | Google Scholar

Appendix

TABLE A1

Table A1. Background of the reviewed studies.