Meta-analyses of differences in blended and traditional learning outcomes and students' attitudes

Abstract

The sudden outbreak of COVID-19 has made blended learning widely accepted, followed by many studies committed to blended learning outcomes and student attitudes. Few studies have, however, focused on the summarized effect of blended learning. To complement this missing link, this study meta-analytically reviews blended learning outcomes and student attitudes by including 30 peer-reviewed journal articles and 70 effect sizes. It concludes that blended learning outcomes are significantly higher than the traditional learning outcomes with a medium effect size, and learners hold significantly more positive attitudes toward blended learning than traditional learning with a medium effect size. Blended learning may be promising, and information technology scientists may focus on the development of more advanced and effective devices to improve blended learning effectiveness.

Introduction

Definitions of blended learning

The sudden outbreak of COVID-19 has made blended learning widely accepted (Yu, 2021a) and it has led first generation scholars to engage in online or blended learning (Mates et al., 2021). Blended learning has been rising as a major component of learning methods, with which various definitions of blended learning have been put forward (Yu, 2015). Blended learning is defined as a mixed teaching approach where both online learning and physical face-to-face learning are integrated assisted with mobile or educational technologies (Yu, 2015). Blended learning could not only facilitate learning outcomes but also fill the gap between online learning and physical classroom-based learning supported by information technologies (Vaughan, 2007). Blended learning could also connect learning to working (Bohle-Carbonell et al., 2013) via mobile or information technologies. Therefore, blended learning could involve distributed, decentralized, hybrid, and flexible learning (Leidl et al., 2020).

In general, blended learning cannot be simply summarized as simple blends of several learning methods involving both physical and online learning. Blended learning can integrate multiple components, e.g., learning situations where online, physical and working situations can be included; knowledge acquisition mechanisms, e.g., retention and attrition; affective elements in learning, e.g., motivation, satisfaction, encouragement, and cognitive loads; and participants, e.g., teachers, learners, and designers. It could be a complicated blend including the traditional, the for- and in-actions, and experiential factors (Viebig, 2022).

Blended learning outcomes

Information technology-assisted blended learning is adopted in every discipline at all levels, which proves effective to promote and improve learning outcomes (Dutton et al., 2002; Zhang and Yu, 2021). Blended learning, a student-centered approach, encourages students to have stronger self-regulation and assume more responsibilities than those who receive traditional face-to-face instruction (Yick et al., 2019). Blended learning requires students to rigidly adhere to academic schedules and to participate in learning activities autonomously (Sun and Rueda, 2012). Strong self-regulation and self-confidence may increase students' engagement time, reduce their dropout rates, and improve their learning outcomes (Liang and Tsai, 2008).

Recent decades have witnessed many studies committed to information technology-assisted learning, which reported that digital tools, online evaluation, and adaptive online courses could improve students' academic achievements (Brodersen and Melluso, 2017). Information technology tools can provide adaptive instruction for individual learners. Blended individualized learning could meet diverse needs of learners and improve academic achievements (Jiménez et al., 2007; Yu, 2021b). blended learning could produce positive learning outcomes in higher education (Sankar et al., 2022). The online and physical information technologies have added benefits to individualized learning by increasing their engagement and intensifying their interest in blended learning (McCarthy et al., 2020). Teachers and students have benefited a lot from this rising blended learning method, and many studies have been committed to the effect of blended learning (Sharpe et al., 2006).

Blended learning combined teachers' instruction with information technologies, which has received great popularity at the elementary educational level (Christensen et al., 2013). Blended learning integrated traditional face-to-face learning with information technology-supported online learning (Pytash and O'Byrne, 2018), which enabled students to have easy access to learning resources wherever and whenever they feel convenient. The mode of blended learning could provide students, especially those who possibly failed the course, with individual learning strategies and protocols (Macaruso et al., 2020). Teachers could also adopt an individualized instruction by delivering knowledge through the information technology-assisted platform when or where they desire, which could bridge the gap between satisfactory learning outcomes and students' actual performance (Shanahan and Lonigan, 2010). The information technologies could immerse learners in academic activities, stimulate their motivation, and improve their learning outcomes (Repetto et al., 2018). Despite this, there are still some contradictory findings regarding blended learning outcomes. An example is that no significant differences are revealed in learners' writing performance between the blog and paper-based writing methods (Ellison and Wu, 2008). Blended learning outcomes are thus worth further examination.

Student attitudes toward blended learning

Many studies have been devoted to the exploration of student attitudes toward blended learning, most of which reported positive student attitudes. Nursing students receiving blended cardiopulmonary resuscitation (CPR) education held more positive attitudes toward the blended approach CPR than the control group who received traditional education (Moon and Hyun, 2019). Integrated with information technologies, the blended learning approach could provide flexible instructional methods for students of all ages. This individualized instructional method could meet students' various social, academic, and educational needs, enhance their engagement, and improve their attitudes toward blended learning (Arrosagaray et al., 2019).

Students hold positive perceptions and attitudes toward asynchronous blended English grammar learning and instruction assisted with computer tools (Pinto-Llorente et al., 2017). They also positively evaluate blended learning assisted with blogs, wikis, podcasts, and virtual classrooms in a teacher education program at RMIT University, Australia (Robertson, 2008). Furthermore, attitudes toward blended learning such as Person-Centered e-Learning (PCeL) exert an important influence on learning outcomes and motivation of students (Motschnig-Pitrik and Mallich, 2004). Teaching, cognitive, and social presences could also influence students' attitudes toward blended learning according the theory of Community of Inquiry (Yin and Yuan, 2022).

Students generally hold positive attitudes toward information technologies to supplement online learning and blended learning (Yu and Yu, 2019; Yu et al., 2019). Attitudes can play an important role in the acceptance of blended learning. Positive attitudes toward blended learning drive students to accept the instruction integrating information technology-assisted instruction with a face-to-face approach, where competitiveness outweighs cooperation among undergraduates (Hwang and Arbaugh, 2009). Despite the closure of educational institutes, most dental students positively evaluate the virtual learning (Cho and Ganesh, 2022). Attitudes toward blended learning are, however, inconsistent in blended physical education (Tenison and Touger-Decker, 2018). Sport and exercise students prefer the traditional face-to-face education despite the pandemic (Finlay et al., 2022). It is thus necessary to explore student attitudes toward blended learning.

Meta-analytical review studies on blended learning

Numerous studies have been committed to blended learning via meta-analyses. It is analytically reviewed that nursing students' knowledge and skills via blended learning improve significantly more than traditional learning (Li et al., 2019). Although blended learning is generally effective, the effectiveness depends on the learning contexts and the way of the use of information technologies. The technology use also negatively influences blended learning since there are still negative effect sizes. Motivation, attitudes, and warrant consideration may be important factors influencing the blended learning effect (Mahmud, 2018).

Blended learning could lead to significantly better academic achievements than traditional learning in the discipline of health (Liu et al., 2016). Assisted with computer technologies, blended learning could enhance academic achievements in higher education (Bernard et al., 2014). Blended learning could lead to significantly higher academic achievements among STEM-disciplined students than traditional learning (Vo et al., 2017). Furthermore, blended learning could improve interactions and critical thinking abilities (Chang and Yeh, 2021), conducive to positive learning outcomes (Means et al., 2013).

Although many studies have systematically and meta-analytically reviewed the effect of blended learning, few of them have summarized the learning outcomes of and attitudes toward blended learning. Except for overall student attitudes toward blended learning, this meta-analytical review will examine the overall effect of blended learning on learning outcomes. Learning outcomes in this study include English listening, speaking skills, and critical thinking, encouragement of computer science students, student engagement, self-efficacy, motivation, knowledge, online achievement test scores, and writing skills.

Research questions and hypotheses

We proposed two research questions, i.e., (1) What differences in learning outcomes can be established between blended learning and the traditional face-to-face approach? (2) What differences in learner attitudes can be established between blended learning and the traditional face-to-face approach? Research hypotheses tend to be proposed and tested in meta-analyses in the field of education (e.g., Foster, 2021; Katsarov et al., 2021). Thus, we proposed two alternative research hypotheses based on the research questions, i.e., (1) Blended learning outcomes are significantly higher than the traditional learning outcomes; (2) Learners hold significantly more positive attitudes toward blended learning than traditional learning.

Research methods

Literature search

The literature search a systematic literature review based on a strict protocol. We followed a flowchart to search literature (Figure 1). EBSCOhost includes many databases such as Business Source Premier, Newspaper Source, Education Resources Information Center (ERIC), MEDLINE, Regional Business News, Library, Information Science & Technology Abstracts, GreenFILE, Teacher Reference Center, European Views of the Americas: 1493 to 1750, eBook Collection (EBSCOhost), EBSCO eClassics Collection (EBSCOhost), OpenDissertations, Modern Language Association (MLA) Directory of Periodicals, MLA International Bibliography with Full Text, Academic Search Complete, Business Source Complete, and Political Science Complete.

Figure 1

Via Boolean logic/phrase, two independent researchers, experienced lecturers in the research team, entered TI blended learning OR hybrid learning (control OR treatment OR experimental) into EBSCOhost for academic search ranging from 2000 to 2020, and 508 results returned (25 August 2020). They reduced the results to 137 by limiting them to fulltext, academic peer-reviewed journal, academic theoretical journal, journal, and English. Finally, they selected 4 peer-reviewed articles of high quality.

Additionally, they obtained 52 results by searching Springer via “(control OR treatment OR experimental)” AND TI “blended learning OR hybrid learning” within the field of Educational Technology, ranging from 2000 to 2020. After removing 47 results, we selected 5 articles of high quality. We obtained 8 results from Web of Science (WOS) and selected 4 of them.

By keying in [Publication Title: blended learning OR hybrid learning] AND [[All: control] OR [All: treatment] OR [All: experimental]] in Taylor & Francis, they obtained 205 results, ranging from 2000 to 2020. By selecting the subject “education”, we reduced the results to 142. They selected 6 high-quality peer-reviewed articles after perusing the articles.

By keying in “Title abstract, keywords: control OR treatment OR experimental, and Title: blended learning OR hybrid learning” in the database Elsevier ScienceDirect, they obtained 56 results, ranging from 2000 to 2020. After careful reading and screening, they selected 11 of them. Besides, we selected results based on both inclusion and exclusion criteria, as well as The STARLITE tool (Booth, 2006).

Inclusion criteria

The studies will be considered eligible if they: (1) focus on blended learning effect via controlled and randomized comparative design; (2) divide the participants into both control and treatment groups and the control group receives traditional face-to-face learning, while the treatment group receives blended learning; (3) include student attitudes toward blended learning or blended learning outcomes such as test scores, satisfaction, and conceptual understanding; (4) are published in English.; (5) belong to peer-reviewed journal articles.

Exclusion criteria

The studies will be excluded if they: (1) are not related to the comparison between blended and traditional learning; (2) are conference abstracts, editorial articles, review articles or meta-analyses; (3) focus on online learning technologies rather than the use of them in education; (4) cannot provide enough information for meta-analysis even after we correspond with the authors.

Evaluation of the literature

We also used University of West England Framework for Critically Appraising Research Articles (Moule et al., 2003) to evaluate the quality of literature. The star rating system used in the evaluation of the literature includes criteria evaluating different sections of a target paper. For the introduction part, two questions are raised to evaluate the quality, i.e., “Is there a clear statement about the topic being investigated? Is there a clear rationale for the research?” Several questions are formulated to evaluate the quality of the method section including clarity, data collection and analysis, qualitative or quantitative research methods. The paper is also evaluated in terms of an ethical statement. For the results/discussion section, we focus on both quantitative and qualitative explorations. The conclusion section needs to include acknowledgment and recommendations.

We deleted those of lower quality and adopted those of higher quality. The quality was rated with “star” (Ren et al., 2019). The article rated higher than nine stars was considered high-quality literature, seven to eight stars indicating medium quality and less than seven stars indicating lower quality. Different opinions were discussed and decided by a third researcher. Finally, we selected 30 peer-reviewed journal articles (Table 1) from the four online databases.

Results

The result section develops based on the proposed alternative research hypotheses.

• H1. Blended learning outcomes are significantly higher than the traditional learning outcomes

Figure 2 addresses this research hypothesis. Two researchers co-worked to calculate the mean, mean difference, and confidence intervals on the basis of the research procedure. We obtained in total 64 results from 30 studies. Figure 2 presents the results calculated from 30 studies using Stata/MP 14.0.

Figure 2

If the value of I² is larger than 50%, the between-studies variation will be considered heterogeneous (Ren et al., 2019) and we should switch a fixed-effect to a random-effect model. As shown in Figure 2, the between-studies variation is heterogeneous (I² = 93.1%; p < 0.00001), so we apply a random-effect model to the analysis. Means, standard deviations, total numbers of participants and mean differences between blended groups and control groups are indicated, as well as a forest plot on the right. The values of weights indicate the power of an individual study influencing the summarized results. The higher the weight is the more powerful influence an individual study will exert on a summarized result. The studies conducted by Baepler et al. (2014) and Botts et al. (2018) have the highest weight values (3.0%), hence influencing the summarized result to a large extent. By contrast, the studies conducted by Yang (2012), Yang et al. (2013), Monteiro and Morrison (2014), and Shorey et al. (2017) have the lowest value (0%) of weight. Therefore, they exert a minimal influence on the summarized result.

In the middle of the forest plot, the vertical line is referred to as a no-effect line. The result will be considered insignificantly different if the line is touched. The diamond at the bottom of the forest line indicates the summarized result. The former is negatively correlated with the latter. The narrower the diamond is, the stronger confidence the result will have. As shown in Figure 2, the diamond is narrow (d = 0.52), ranging from 0.41 to 0.63 in 95% CI, which indicates that the result has a strong confidence. This result does not touch the no-effect line. The diamond is placed on the right side of the no-effect line, indicating the mean of blended learning outcomes is larger than that of the traditional learning. Therefore, we accept the first research hypothesis that the blended learning outcomes are significantly higher than the traditional learning outcomes.

However, publication bias may exist. As shown in Figure 3, one dot indicates an individual study. The unbiased distribution of dots is symmetrically aligned on both sides of the middle line. Studies on blended-to-traditional learning outcomes are, however, asymmetrically distributed to the right of the middle line. This indicates that more positive results might have been published than negative ones (Egger's test coefficient = 2.29, t = 4.02, p = 0.00; Begg's test z = 4.02, p = 0.00). The sensitivity analysis (Figure 4) shows that no individual study influences the pooled effect size since all of the meta-analysis estimates exist between the lower and upper CI limits given a named study is omitted. This indicates that the results of this meta-analysis are stable.

• H2. Learners hold significantly more positive attitudes toward blended learning than traditional learning

Figure 3

Figure 4

Figure 5 addresses the second research hypothesis. Similarly, both researchers cooperated to calculate the mean, mean difference, and confidence intervals according to the research design. We finally obtained in total 4 results from 4 studies. Figure 5 presents the results calculated from the 4 studies. We apply a random-effect model to the analysis since the between-studies variation is heterogeneous (I² = 86.1%; p = 0.004). We retrieved the data from four studies to address the question, where the data provided by Chang et al. (2014) has the heaviest weight (44.9%), exerting the greatest influence on the summarized result. On the contrary, the study conducted by Shorey et al. (2017) has the lightest weight (0.0%), exerting a minimal influence on the summarized result.

Figure 5

The diamond summarizing the final result is narrow (d = 0.57), ranging from 0.19 to 0.96 in 95% CI. This indicates that the summarized result is reliable and convincing. The summarized result does not touch the vertical no-effect line and the result is placed on the right side. This implies that the mean of blended learning is significantly larger than that of the traditional learning. Therefore, we accept the second research hypothesis that learners hold significantly more positive attitudes toward blended learning than traditional learning.

We did not find publication bias. Studies on attitudes toward blended learning are symmetrically distributed along both sides of the middle line (Figure 6). This means that more positive results may have been published than negative ones (Egger's test coefficient = 2.29, t = 0.45, p = 0.679, 95% CI = −11.99, 16.58; Begg's test z = 0.19, p = 0.581). We also carried out a sensitivity analysis (Figure 7), revealing that the meta-analytical results are stable since the meta-analysis estimates are all positioned between the lower and upper CI limits given a name study is omitted.

Figure 6

Figure 7

Calculation of effect sizes

The Cohen's d effect size gains popularity in studies of social sciences. The effect size is considered small if d < 0.2, medium if 0.5 < d < 0.8, large if d > 0.8 (Cohen, 1988). We calculated the effect sizes via Cohen's d formula. The effect sizes were obtained through the standardized mean differences divided by the pooled standard deviation across both treatment and control groups. We obtained 74 effect sizes from the selected 30 peer-reviewed journal articles (Table 2).

As for blended learning outcomes, the study of Gordon et al. (2005) obtained a large effect size [d = 7.33 (95% CI: 6.42, 8.24)], while the study of Monteiro and Morrison (2014) gained the smallest effect size [d = 0.00 (95% CI: −0.5, 0.5)]. Most of the other studies obtained medium effect size, e.g., Baepler et al. (2014) [d = 0.52 (95% CI: 0.32, 0.71)], Hill et al. (2017) [d = 0.52 (95% CI: 0.11, 0.94)], and Botts et al. (2018) [d = 0.53 (95% CI: 0.30, 0.76)]. The total effect size is at the medium level [d = 0.522 (95% CI: 0.410, 0.634)].

As for student attitudes toward blended learning, Al-Qatawneh et al. (2020) obtained a large effect size [d = 1.42 (95% CI: 0.96, 1.87)], while McCutcheon et al. (2018) obtained a medium effect size [d = 0.61 (95% CI: 0.25, 0.98)], and Chang et al. (2014) obtained a small size [d = 0.26 (95% CI: −0.22, 0.75)]. So did Shorey et al. (2017) [d = 0.26 (95% CI: 0.01, 0.51)]. The total effect size is at the medium level [d = 0.63 (95% CI: 0.14, 1.11)] (Table 3).

Discussion

Blended learning, an approach gaining high popularity, has also been categorized into the “third generation” of information technology-aided education (Phipps and Merisotis, 1999, p. 26). The first generation means correspondence education via a one-way pedagogical approach, which adopts traditional tools such as radio, television, broadcaster, and mail. The second generation develops into distance education with the help of traditional instructional technologies, e.g., multimedia projecting systems and computer software assisted boards. The third generation, conducive to higher education (Phipps and Merisotis, 1999, p. 26), evolves into an advanced stage, featuring various blends of information technologies, face-to-face learning methods, opinion sharing platforms, communicative platforms, and discussion forums. This third generation takes advantage of many benefits of the information technologies and optimizes the advantages of various instructional approaches. It is, therefore, reasonable to find that blended learning can lead to higher learning outcomes than traditional learning.

Blended learning can offset the disadvantages of online learning and traditional face-to-face learning. Online learning cannot be applied to all kinds of courses. Some of them may be appropriate for traditional learning, while others may obtain benefits from online learning. The mode of instruction may vary on the basis of instructional purposes of the specific course (Bolliger and Martindale, 2004). Blended learning combines online with traditional face-to-face learning and thus compensates for the disadvantages and expands the advantages (Bersin, 2004). Online learning may cause lower engagement and higher dropout rates, while the traditional learning may give rise to inconvenience in time and space of instruction. Blended learning can minimize these drawbacks by flexibly arranging learning activities and frequently encouraging students to engage in learning (Singh, 2003). Blended learning can also cater for different learners accustomed to various learning styles, which has led blended learning to be widely accepted (Bonk, 2006). This individualized feature of blended learning has enabled it to be a tendency of the pedagogical approach, which may also have caused a positive student attitude toward blended learning.

Blended with information technologies such as blog, twitter, Facebook, and messenger, learners may feel free and easy to express their opinions because keying in the discussion forum is more relaxing and motivating than paper-based writing (Kitchakarn, 2012). Assisted with blogs in writing, learners may improve their reflection compared with the traditional writing on printed paper (Harland and Wondra, 2011). These findings may account for why participants hold positive attitudes toward blog-based writing rather than paper-based writing (Arslan, 2014). Students may hold positive attitudes toward the effectiveness of blogs in learning because they believe they can conveniently access various viewpoints and increase the engagement in writing practice (Ellison and Wu, 2008).

The rationales for favorable attitudes toward blended learning may be of variety in the information age. The blended learning integrates information technologies into learning. Students can obtain a sea of academic resources through their mobile devices conveniently. They can also review what they feel interested in wherever and whenever they feel convenient. They can share opinions with peers and teachers through the application installed in their mobile devices, record their voices via recorders, or send messages, voices, or videos to anybody they desire to share. They can also review their rankings via the calculation function of the information technology, which may display their learning profile and encourage them to learn.

The information technology-blended approach can also catch learners' attention by displaying colorful pictures or animate videos. Thus, their satisfaction will be enhanced, coupled with favorable attitudes and improved learning outcomes. Blended learning can also facilitate learning effectiveness and speed up their knowledge acquisition, which will potentially cultivate the favorable attitudes. However, excessive exposure to the blended learning approach may bring about distractions, which may explain the phenomenon that blended learning outcomes sometimes remain insignificantly different from traditional learning.

While this study revealed positive blended learning outcomes and students' attitudes by means of meta-analysis and rigid design, the blended process might need further exploration. Given that blended learning can be implemented in several different ways, comparing only outcomes without considering the actual processes that led to those outcomes can be quite problematic. The flipped pedagogical approach has been frequently integrated into the blended learning methods. The lectures are moved out of class that is implemented online before class. The in-class lecturing has been converted to academic activities under the guidance of the lecturer. After class, students can interact with peers and teachers through the online communicative applications (Baepler et al., 2014). Another model of blended methods was implemented, which included five features beneficial to learning and teaching effectiveness (McCarthy et al., 2020). Thus, the outcomes of the study could be enhanced by including the implementation process of blended methods.

Conclusion

This concluding section will summarize major findings and limitations of this study, together with future research directions.

Major findings

After meta-analysis and calculation of effect sizes, this study concludes that blended learning outcomes are significantly higher than the traditional learning outcomes with a medium effect size, and learners hold significantly more positive attitudes toward blended learning than traditional learning with a medium effect size.

Findings in this study are generally consistent with previous studies. The blended pedagogical approach could improve nurses' research abilities and critical thinking and lead to positive outcomes (Chen et al., 2021). Blended learning could satisfy the urgent needs of education during the COVID-19 pandemic and in the future (Ahmed and Opoku, 2021). Most of students held positive attitudes toward blended learning and teaching although they desired to change the role of teachers and students, as well as the method of knowledge delivery (Zarrinfard et al., 2021). The blended delivery of knowledge could increase the interactions and learning opportunities among students who could then be motivated and receive flexible education (Bashir et al., 2021).

Limitations

Although this study is rigidly designed, several limitations still exist. Firstly, the publishers may tend to publish those with positive results, which may have caused publication bias. Secondly, the study selects publications merely in English language, which may not cover those written in other languages. Thirdly, the library resources are limited, which may not include all the related publications.

Future research directions

Blended learning may be promising especially during this COVID-19 pandemic time, and information technology scientists may focus on the development of more advanced and effective devices to improve blended learning effectiveness. Various factors such as perceived learning effectiveness, perceived usefulness and easiness, social presence, and demographic variables may be considered when we design blended learning technologies in the future. Future researchers could also enhance students' emotional intelligence and cognitive engagement, leading to favorable learning habits (Iqbal et al., 2022).

Funding

This work is supported by 2019 MOOC of Beijing Language and Culture University (MOOC201902) (Important) Introduction to Linguistics; Introduction to Linguistics of online and offline mixed courses in Beijing Language and Culture University in 2020; Special fund of Beijing Co-construction Project-Research and reform of the Undergraduate Teaching Reform and Innovation Project of Beijing higher education in 2020-innovative multilingual + excellent talent training system (202010032003); The research project of Graduate Students of Beijing Language and Culture University Xi Jinping: The Governance of China (SJTS202108).

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

• 1

  AhmedV.OpokuA. (2021). Technology supported learning and pedagogy in times of crisis: the case of COVID-19 pandemic. Educ. Info. Technol.27, 365–405. 10.1007/s10639-021-10706-w

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 2

  Al-QatawnehS.EltahirM. E.AlsalhiN. R. (2020). The effect of blended learning on the achievement of HDE students in the methods of teaching Arabic language course and their attitudes towards its use at Ajman University: a case study. Educ. Info. Technol.25, 2101–2127. 10.1007/s10639-019-10046-w

  • CrossRef
  • Google Scholar

• 3

  ArrosagarayM.Gonzalez-PeiteadoM.Pino-JusteM.Rodriguez-LopezB. (2019). A comparative study of Spanish adult students' attitudes to ICT in classroom, blended and distance language learning modes. Comput. Educ.134, 31–40. 10.1016/j.compedu.2019.01.016

  • CrossRef
  • Google Scholar

• 4

  ArslanR. S. (2014). Integrating feedback into prospective English language teachers' writing process via blogs and portfolios. TOJET13, 131–150.

  • Google Scholar

• 5

  BaeplerP.WalkerJ. D.DriessenM. (2014). It's not about seat time: blending, flipping, and efficiency in active learning classrooms. Comput. Educ.78, 227–236. 10.1016/j.compedu.2014.06.006

  • CrossRef
  • Google Scholar

• 6

  BashirA.BashirS.RanaK.LambertP.VernallisA. (2021). Post-COVID-19 adaptations; the shifts towards online learning, hybrid course delivery and the implications for biosciences courses in the higher education setting. Front. Educ.6, 711619. 10.3389/feduc.2021.711619

  • CrossRef
  • Google Scholar

• 7

  BazelaisP.DoleckT. (2018). Blended learning and traditional learning: a comparative study of college mechanics courses. Educ. Info. Technol.23, 2889–2900. 10.1007/s10639-018-9748-9

  • CrossRef
  • Google Scholar

• 8

  BernardR. M.BorokhovskiE.SchmidR. F.TamimR. M.AbramiP. C. (2014). A meta-analysis of blended learning and technology use in higher education: from the general to the applied. J. Comput. High. Educ.26, 87–122. 10.1007/s12528-013-9077-3

  • CrossRef
  • Google Scholar

• 9

  BersinJ. (2004). The Blended Learning Book: Best Practices, Proven Methodologies, and Lessons Learned. San Francisco, CA: Pfeiffer.

  • Google Scholar

• 10

  Bohle-CarbonellK.Dailey-HebertA.GerkenM.GrohnertT. (2013). “Supporting learner engagement through problem-based learning: institutional and instructional implications,” in Increasing Student Engagement and Retention in e-learning Environments: Web 2.0 and Blended Learning Technologies, vol. 7, eds P. Blessinger and C. Wankel (Emerald).

  • Google Scholar

• 11

  BolligerD. U.MartindaleT. (2004). Keys factors for determining student satisfaction in online courses. Int. J. E-Learn.3, 61–67.

  • Google Scholar

• 12

  BonkC. J. (2006). The future of online teaching and learning in higher education. Educause Q.11, 22–30. 10.4018/978-1-7998-0062-0.ch003

  • CrossRef
  • Google Scholar

• 13

  BoothA. (2006). “Brimful of STARLITE” toward standards for reporting literature searches. J. Med. Lib. Assoc.94, 421–429.

  • Pubmed Abstract
  • Google Scholar

• 14

  BottsR. T.CarterL.CrockettC. (2018). Using the blended learning approach in a quantitative literacy course. PRIMUS28, 236–265. 10.1080/10511970.2017.1371264

  • CrossRef
  • Google Scholar

• 15

  BrodersenR. M.MellusoD. (2017). Summary of Research on Online and Blended Learning Programs that Offer Differentiated Learning Options. Available online at: https://files.eric.ed.gov/fulltext/ED572935.pdf (accessed April 20, 2022).

  • Google Scholar

• 16

  ChangC. C.ShuK. M.LiangC.TsengJ. S.HsuY. S. (2014). Is blended e-learning as measured by an achievement test and self-assessment better than traditional classroom learning for vocational high school students?Int. Rev. Res. Open Dist. Learn.15, 213–231. 10.19173/irrodl.v15i2.1708

  • CrossRef
  • Google Scholar

• 17

  ChangW. L.YehY. C. (2021). A blended design of game-based learning for motivation, knowledge sharing and critical thinking enhancement. Technol. Pedagogy Educ.30, 271–285. 10.1080/1475939X.2021.1885482

  • CrossRef
  • Google Scholar

• 18

  ChenQ. R.TangS. Y.LiuD.ZhouC. Y.CastroA. R.JiangS.et al. (2021). Effects of a blended emergent research training programme for clinical nurses on nursing research competence and critical thinking (part 2): a quasi-experimental study. J. Clin. Nurs.31, 755–769. 10.1111/jocn.15934

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 19

  ChoA.GaneshN. (2022). Dental students' perception of a blended learning approach to clinic orientation. J. Dent. Educ.86, 721–725. 10.1002/jdd.12875

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 20

  ChristensenC. M.HornM. B.StakerH. (2013). Is K-12 Blended Learning Disruptive? An Introduction to the Theory of Hybrids. Available online at: https://www.christenseninstitute.org/wp-content/uploads/2014/06/Is-K-12-blended-learning-disruptive.pdf (accessed April 20, 2022).

  • Google Scholar

• 21

  CohenJ. (1988). Statistical Power Analysis for the Behavioral Sciences, 2nd Edn. Hillsdale, NJ: L. Erlbaum Associates.

  • Google Scholar

• 22

  CortizoJ. L.RodriguezE.VijandeR.SierraJ. M.NoriegaA. (2010). Blended learning applied to the study of mechanical couplings in engineering. Comput. Educ.54, 1006–1019. 10.1016/j.compedu.2009.10.006

  • CrossRef
  • Google Scholar

• 23

  DuttonJ.DuttonM.PerryJ. (2002). How do online learning students differ from lecture students?J. Asynchronous Learn. Networks6, 1–20. 10.24059/olj.v6i1.1869

  • CrossRef
  • Google Scholar

• 24

  EllisonN.WuY. (2008). Blogging in the classroom: a preliminary exploration of student attitudes and impact on comprehension. J. Educ. Multimed. Hypermed.17, 99–122.

  • Google Scholar

• 25

  FinlayM. J.TinnionD. J.SimpsonT. (2022). A virtual versus blended learning approach to higher education during the COVID-19 pandemic: the experiences of a sport and exercise science student cohort. J. Hosp. Leis. Sport Tour. Educ.30, 100363. 10.1016/j.jhlste.2021.100363

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 26

  FosterC. (2021). Implementing confidence assessment in low-stakes, formative mathematics assessments. Int. J. Sci. Math. Educ.3, 1–9. 10.1007/s10763-021-10207-9

  • CrossRef
  • Google Scholar

• 27

  GongJ.RuanM.YangW.PengM.WangZ.OuyangL.et al. (2021). Application of blended learning approach in clinical skills to stimulate active learning attitudes and improve clinical practice among medical students. PeerJ9, e11690. 10.7717/peerj.11690

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 28

  GordonD. L.IssenbergS. B.GordonM. S.LacombeD.MCGaghieW. C.PetrusaE. R. (2005). Stroke training of prehospital providers: an example of simulation-enhanced blended learning and evaluation. Med. Teach.27, 114–121. 10.1080/01421590400029756

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 29

  HarlandD. J.WondraJ. D. (2011). Preservice teachers' reflection on clinical experiences: a comparison of blog and final paper assignments. J. Digital Learn. Teach. Educ.27, 128–133. 10.1080/21532974.2011.10784669

  • CrossRef
  • Google Scholar

• 30

  HillT.ChidambaramL.SummersJ. D. (2017). Playing ‘catch up' with blended learning: performance impacts of augmenting classroom instruction with online learning. Behav. Info. Technol.36, 54–62. 10.1080/0144929X.2016.1189964

  • CrossRef
  • Google Scholar

• 31

  HwangA.ArbaughJ. B. (2009). Seeking feedback in blended learning: competitive versus cooperative student attitudes and their links to learning outcome. J. Comput. Assist. Learn.25, 280–293. 10.1111/j.1365-2729.2009.00311.x

  • CrossRef
  • Google Scholar

• 32

  IqbalJ.AsgharM. Z.AshrafM. A.YiX. (2022). The impacts of emotional intelligence on students' study habits in blended learning environments: the mediating role of cognitive engagement during COVID-19. Behav. Sci.12, 14. 10.3390/bs12010014

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 33

  JiaJ.ChenY.DingZ.RuanM. (2012). Effects of a vocabulary acquisition and assessment system on students' performance in a blended learning class for English subject. Comput. Educ.58, 63–76. 10.1016/j.compedu.2011.08.002

  • CrossRef
  • Google Scholar

• 34

  JiménezT. C.GrafV. L.RoseE. (2007). Gaining access to general education: the promise of universal design for learning. Issues Teach. Educ.16, 41–54.

  • Google Scholar

• 35

  KatsarovJ.AndornoR.KromA.van den HovenM. (2021). Effective strategies for research integrity training—a meta-analysis. Educ. Psychol. Rev.27, 1–21. 10.1007/s10648-021-09630-9

  • CrossRef
  • Google Scholar

• 36

  KitchakarnO. (2012). Using blogs to improve students' summary writing abilities. Turk. Online J. Dist. Educ.13, 209–219.

  • Google Scholar

• 37

  LeidlD. M.RitchieL.MoslemiN. (2020). Blended learning in undergraduate nursing education - a scoping review. Nurse Educ. Today86, 104318. 10.1016/j.nedt.2019.104318

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 38

  LiC.HeJ.YuanC. X.ChenB.SunZ. L. (2019). The effects of blended learning on knowledge, skills, and satisfaction in nursing students: a meta-analysis. Nurse Educ. Today82, 51–57. 10.1016/j.nedt.2019.08.004

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39

  LiangJ. C.TsaiC. C. (2008). Internet self-efficacy and preferences towards constructivist internet-based learning environments: a study of pre-school teachers in Taiwan. Educ. Technol. Soc.11, 226–237.

  • Google Scholar

• 40

  LiuQ.PengW. J.ZhangF.HuR.LiY. X.YanW. R. (2016). The effectiveness of blended learning in health professions: systematic review and meta-analysis. J. Med. Internet Res.18, e2. 10.2196/jmir.4807

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 41

  Lopez-PerezM. V.Perez-LopezM. C.Rodriguez-ArizaL.Argente-LinaresE. (2013).The influence of the use of technology on student outcomes in a blended learning context. ETRandD-Educ. Technol. Res. Dev.61, 625–638. 10.1007/s11423-013-9303-8

  • CrossRef
  • Google Scholar

• 42

  MaX.MaX.LiL.LuoX.ZhangH.LiuY. (2021). Effect of blended learning with BOPPPS model on Chinese student outcomes and perceptions in an introduction course of health services management. Adv. Physiol. Educ.45, 409–417. 10.1152/advan.00180.2020

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 43

  MacarusoP.WilkesS.PrescottJ. E. (2020). An investigation of blended learning to support reading instruction in elementary schools. Educ. Technol. Res. Dev.68, 2839–2852. 10.1007/s11423-020-09785-2

  • CrossRef
  • Google Scholar

• 44

  MahmudM. M. (2018). Technology and language - what works and what does not: a meta-analysis of blended learning research. J. Asia TEFL15, 365–382. 10.18823/asiatefl.2018.15.2.7.365

  • CrossRef
  • Google Scholar

• 45

  MatesL.MillicanA.HansonE. (2021). Coping with COVID; understanding and mitigating disadvantages experienced by first generation scholars studying online. Br. J. Educ. Stud.12, 1–22. 10.1080/00071005.2021.1966382

  • CrossRef
  • Google Scholar

• 46

  McCarthyE. M.LiuY.SchauerK. L. (2020). Strengths-based blended personalized learning: an impact study using virtual comparison group. J. Res. Technol. Educ.52, 353–370. 10.1080/15391523.2020.1716202

  • CrossRef
  • Google Scholar

• 47

  McCutcheonK.O'HalloranP.LohanM. (2018). Online learning versus blended learning of clinical supervisee skills with pre-registration nursing students: a randomised controlled trial. Int. J. Nurs. Stud.82, 30–39. 10.1016/j.ijnurstu.2018.02.005

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 48

  MeansB.ToyamaY.MurphyR.BakiM. (2013). The effectiveness of online and blended learning: a meta-analysis of the empirical literature. Teach. Coll. Rec.115, 030303. 10.1177/016146811311500307

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 49

  MonteiroE.MorrisonK. (2014). Challenges for collaborative blended learning in undergraduate students, Educ. Res. Eval.20, 564–591, 10.1080/13803611.2014.997126

  • CrossRef
  • Google Scholar

• 50

  MoonH.HyunH. S. (2019). Nursing students' knowledge, attitude, self-efficacy in blended learning of cardiopulmonary resuscitation: a randomized controlled trial. BMC Med. Educ.19, 414. 10.1186/s12909-019-1848-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 51

  Motschnig-PitrikR.MallichK. (2004). Effects of person-centered attitudes on professional and social competence in a blended learning paradigm. Educ. Technol. Soc.7, 176–192.

  • Google Scholar

• 52

  MouleP.PontinD.Gilchrist.MIngramR. (2003). Critical Appraisal Framework. Available online at: http://learntech.uwe.ac.uk/da/Default.aspx?pageid=1445 (accessed August 4, 2021).

  • Google Scholar

• 53

  MuellerC.MildenbergerT.LübckeM. (2020). Do we always need a difference? Testing equivalence in a blended learning setting. Int. J. Res. Method Educ.43, 283–295. 10.1080/1743727X.2019.1680621

  • CrossRef
  • Google Scholar

• 54

  OlitskyN. H.CosgroveS. B. (2014). The effect of blended courses on student learning: evidence from introductory economics courses. Int. Rev. Econ. Educ.15, 17–31. 10.1016/j.iree.2013.10.009

  • CrossRef
  • Google Scholar

• 55

  Pérez-MarínD.Pascual-NietoI. (2012). A case study on the use of blended learning to encourage computer science students to study. J. Sci. Educ. Technol.21, 74–82. 10.1007/s10956-011-9283-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 56

  PhippsR.MerisotisJ. (1999). What's the Difference? A Review of Contemporary Research on the Effectiveness of Distance Learning in Higher Education. Washington, DC: The Institute for Higher Education Policy.

  • Google Scholar

• 57

  Pinto-LlorenteA. M.Sanchez-GomezM. C.Garcia-PenalvoF. J.Casillas-MartinS. (2017). Students' perceptions and attitudes towards asynchronous technological tools in blended-learning training to improve grammatical competence in English as a second language. Comput. Human Behav.72, 632–643. 10.1016/j.chb.2016.05.071

  • CrossRef
  • Google Scholar

• 58

  PytashK. E.O'ByrneW. I. (2018). “Research on literacy instruction and learning in virtual, blended, and hybrid environments,” in Handbook of Research on K-12 Online and Blending Learning, 2nd Edn, eds K. Kennedy and R. E. Ferdig (Pittsburgh, PA: Carnegie Mellon University Press/ETC Press), 303–314.

  • Google Scholar

• 59

  RenZ. J.CaoD. H.RenP. W.YangB.LuD. L.LiaoJ.et al. (2019). Aryl hydrocarbon receptor rs2066853 gene polymorphisms and male infertility risk: a meta-analysis. Ren. Fail.41, 987–994. 10.1080/0886022X.2019.1673775

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 60

  RepettoJ. B.SpitlerC. J.CoxP. R. (2018). “Research on at-risk learners in K-12 online learning,” in Handbook of Research on K-12 Online and Blending Learning, 2nd Edn, eds K. Kennedy and R. E. Ferdig (Pittsburgh, PA: Carnegie Mellon University Press), 107–134.

  • Google Scholar

• 61

  RobertsonI. (2008). Learners' attitudes to wiki technology in problem based, blended learning for vocational teacher education. Australas. J. Educ. Technol.24, 425–441. 10.14742/ajet.1202

  • CrossRef
  • Google Scholar

• 62

  SankarJ. P.KalaichelviR.ElumalaiK. V.AlqahtaniM. S. M. (2022). Effective blended learning in higher education during COVID-19. Info. Technol. Learn. Tools88, 214–228. 10.33407/itlt.v88i2.4438

  • CrossRef
  • Google Scholar

• 63

  ShanahanT.LoniganC. J. (2010). The national early literacy panel. Educ. Res.39, 279–285. 10.3102/0013189X10369172

  • CrossRef
  • Google Scholar

• 64

  SharpeR.BenfieldG.RobertsG.FrancisR. (2006). The Undergraduate Experience of Blended e-learning: A Review of UK Literature And Practice. The Higher Education Academy, Available online at: http://www.heacademy.ac.uk/assets/documents/research/literature_reviews/blended_elearning_exec_summary_1.pdf (accessed April 20, 2022).

  • Google Scholar

• 65

  ShoreyS.KowitlawakulY.Kamala DeviM.ChenH. C.SoongS. K. A.AngE. (2017). Blended learning pedagogy designed for communication module among undergraduate nursing students: a quasi-experimental study. Nurse Educ. Today 120–126. 10.1016/j.nedt.2017.11.011

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 66

  SinghH. (2003). Building effective blended learning programs. Educ. Technol.43, 51–54. 10.4018/978-1-7998-7607-6.ch002

  • CrossRef
  • Google Scholar

• 67

  SuanaW.DistrikI. W.HerlinaK.MahartaN.PutriN. M. A. A. (2019). Supporting Blended Learning Using Mobile Instant Messaging Application: its Effectiveness and Limitations. Int. J. Instr.12, 1011–1024. 10.29333/iji.2019.12165a

  • CrossRef
  • Google Scholar

• 68

  SunJ.RuedaR. (2012). Situational interest, computer self-efficacy and self-regulation: their impact on student engagement in distance education. Br. J. Educ. Technol.43, 191–204. 10.1111/j.1467-8535.2010.01157.x

  • CrossRef
  • Google Scholar

• 69

  TenisonE.Touger-DeckerR. (2018). Impact of e-learning or blended learning versus face-to-face learning in regard to physical examination skills, knowledge, and attitudes among health professions students. Top. Clin. Nutr.33, 259–270. 10.1097/TIN.0000000000000149

  • CrossRef
  • Google Scholar

• 70

  ThaiN. T. T.De WeverB.ValckeM. (2017). The impact of a flipped classroom design on learning performance in higher education: looking for the best “blend” of lectures and guiding questions with feedback. Comput. Educ.107, 113–126. 10.1016/j.compedu.2017.01.003

  • CrossRef
  • Google Scholar

• 71

  VaughanN. (2007). Perspectives on blended learning in higher education. Int. J. E-Learn.6, 81–94.

  • Google Scholar

• 72

  ViebigC. (2022). Blended learning in entrepreneurship education: a systematic literature review. Educ. Train.64, 533–558. 10.1108/ET-05-2021-0164

  • CrossRef
  • Google Scholar

• 73

  VoH. M.ZhuC.DiepN. A. (2017). The effect of blended learning on student performance at course-level in higher education: a meta-analysis. Stud. Educ. Eval.53, 17–28. 10.1016/j.stueduc.2017.01.002

  • CrossRef
  • Google Scholar

• 74

  WangY. F.TuC.-M.HsuL. (2020). Learning outcomes of a blended learning system for green food and beverage education. Int. J. Mob. Blended Learn.12, 66–78. 10.4018/IJMBL.2020070105

  • CrossRef
  • Google Scholar

• 75

  YangY. F. (2012). Blended learning for college students with English reading difficulties. Comput. Assist. Lang. Learn.25, 393–410. 10.1080/09588221.2011.597767

  • CrossRef
  • Google Scholar

• 76

  YangY. T. C.ChuangY. C.LiL. Y.TsengS. S. (2013). A blended learning environment for individualized English listening and speaking integrating critical thinking. Comput. Educ.63, 285–305. 10.1016/j.compedu.2012.12.012

  • CrossRef
  • Google Scholar

• 77

  YehY. C.HuangL. Y.YehY. L. (2011). Knowledge management in blended learning: effects on professional development in creativity instruction. Comput. Educ.56, 146–156. 10.1016/j.compedu.2010.08.011

  • CrossRef
  • Google Scholar

• 78

  YenJ. C.LeeC. Y. (2011). Exploring problem solving patterns and their impact on learning achievement in a blended learning environment. Comput. Educ.56, 138–145. 10.1016/j.compedu.2010.08.012

  • CrossRef
  • Google Scholar

• 79

  YickK. L.YipJ.AuS. C.LaiY. Y.YuA. (2019). Effectiveness of blended learning in the first year of fashion education. Int. J. Fash. Des. Technol. Educ.12, 178–188. 10.1080/17543266.2018.1546910

  • CrossRef
  • Google Scholar

• 80

  YinB.YuanC. H. (2022). Blended learning performance influence mechanism based on community of inquiry. Asia Pac. J. Educ.11, 1–16. 10.1080/02188791.2022.2061912

  • CrossRef
  • Google Scholar

• 81

  YuZ. (2015). Blended learning over two decades. Int. J. Info. Commun. Technol. Educ.11, 1–19. 10.4018/IJICTE.2015070101

  • CrossRef
  • Google Scholar

• 82

  YuZ. (2021a). The effects of gender, educational level, and personality on online learning outcomes during the COVID-19 pandemic. Int. J. Educ. Technol. High. Educ.18, 1–17. 10.1186/s41239-021-00252-3

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 83

  YuZ. (2021b). The effect of teacher presence in videos on intrinsic cognitive loads and academic achievements. Innov. Educ. Teach. Int.18, 1–12. 10.1080/14703297.2021.1889394

  • CrossRef
  • Google Scholar

• 84

  YuZ.YuX. (2019). An extended technology acceptance model of a mobile learning technology. Comput. Appl. Eng. Educ.27, 721–732. 10.1002/cae.22111

  • CrossRef
  • Google Scholar

• 85

  YuZ.ZhuY.YangZ.ChenW. (2019). Student satisfaction, learning outcomes, and cognitive loads with a mobile learning platform. Comput. Assist. Lang. Learn.32, 323–341. 10.1080/09588221.2018.1517093

  • CrossRef
  • Google Scholar

• 86

  ZarrinfardS.RahimiM.MohsenyA. (2021). Flipping an on-campus general English course: a focus on technology complexity of instruction and learners' levels of impulsivity. Int. J. Educ. Technol. High. Educ.18, 1–25. 10.1186/s41239-021-00280-z

  • CrossRef
  • Google Scholar

• 87

  ZhangQ.YuZ. (2021). A literature review on the influence of Kahoot! on learning outcomes, interaction, and collaboration. Educ. Info. Technol.26, 4507–4535. 10.1007/s10639-021-10459-6

  • CrossRef
  • Google Scholar

• 88

  ZhouC. Y. (2018). Empirical study on the effectiveness of teaching model of college English Writing within blended learning mode. Educ. Sci.: Theory Pract.18, 1060–1076. 10.12738/estp.2018.5.009

  • CrossRef
  • Google Scholar