Big Data Analysis of a Dedicated YouTube Channel as an Open Educational Resource in Hand Surgery

Introduction

YouTube, the world’s largest media-sharing site, with nearly 2 billion users [1], was launched in 2005 and bought by Google in 2006. It is one of the most popular internet sites and also the second most popular search engine used after Google [2]. It is also a large repository of health professional education videos with dedicated channels on various educational subject matter with numerous regular subscribers [3–7]. These online educational videos that are universally and freely accessible to all learners are termed open educational resources (OER). They facilitate learning by using the dual-channel of auditory and visual pathways, founded on the cognitive theory of multimedia learning of Mayer, which, when combined with motivation, further improves the learning process [8]. OER builds generative learning - the process of learning and building understanding through generating relationships, connections, and associations between the dual stimuli of the new knowledge and existing knowledge [9]. These videos are also known as reusable learning objects as they can be reused and repurposed [10].

Despite the many web-based platforms to share educational materials, YouTube remains one of the most accessed platforms for training videos. Despite its usefulness, there have been doubts about the quality of the videos as many have been uploaded without any quality assessment [11, 12]. Surgical educators who host YouTube channels to teach surgery have found it easy to upload their videos. YouTube offers many advantages such as analytics, social media sharing and streaming videos and a wide audience. It provides basic feedback tools for viewers to vote on whether they liked a video and add comments. Most of the videos on YouTube are ranked on popularity, the number of views, and comments. The analytics could be used to assess uptake and to improve the teaching materials if the comments were specific to the surgery topics [13, 14].

Postgraduate surgical education is evolving due to changes in the healthcare system that have impacted clinical training. Many surgical trainees must be proactive in their training because of missed opportunities and accept the new norm of learning outside the operating theater, whether it is through simulation training, reading textbooks, or using online resources [15].

The use of online videos is a novel way to introduce and teach trainees surgical procedures. Technology has revolutionized surgical education owing to its advanced multimedia capabilities. Surgical trainees are using social media platforms such as Facebook, Twitter, and YouTube as learning spaces. Internet access, especially with the broader bandwidths, has made online learning more accessible, and most trainees are familiar with portable electronic devices. It has removed geographical barriers, and the trainees can readily access online resources like journals and watch surgical videos from wherever the procedures are performed [16, 17].

Educational videos explain difficult concepts using graphic diagrams, dynamic illustrations, and simulation, with pauses to demonstrate the anatomical landmarks or highlight the more complicated steps [18]. Multimedia-driven learning has been shown by Friedl and colleagues to significantly improve trainees’ clinical reasoning, technical knowledge, and performance of complex three-dimensional surgical procedures [19]. These instructional videos will benefit the trainees if they are well-designed, concise, clearly presented, scientifically correct, and meet their learning needs. They are useful for reference, especially for learning specific techniques for the rarely performed surgical procedures. Trainees and experienced surgeons can review these videos prior to their own surgery and watch how other colleagues perform them [12, 18].

Theoretical Framework

The use of videos for instructional learning is founded on the following theories and frameworks.

Firstly, the information processing model describes how an individual learns through the reception and assimilation of stimulus through the sensory organs (Figure 1). The sensory modalities involved in the acquisition of new knowledge and skills by a learner are generally through the visual modality, the auditory modality, and the haptic feedback through the kinaesthetic modality [20].

FIGURE 1

FIGURE 1. Information processing model.

The practical evidence for multimedia learning is that adding images and animation to words would improve how students understand and retain content [21]. It is based on three principles:

1. Dual-channel principle–states that most learners learn through an auditory/verbal channel (information provided using text as the spoken word, and as the learners read the text, they hear the words) and a pictorial/visual channel (information provided through the use of images, still and moving, which the learners see). Both channels interact during the acquisition of knowledge in working memory.

2. Limited capacity principle–states that due to the limitation of working memory to simultaneously process several pieces of information, only a fraction of what a person takes in through both acquisition channels can be retained in the working memory.

3. Active processing principle–states that the learners’ have an innate tendency to reference themselves and their previous experiences in selecting and processing new information. This is done by activating and integrating prior knowledge stored in the long-term memory and then processing the new selected and organized information in the working memory.

The theoretical framework for the use of instructional videos is founded on Mayer’s Cognitive Theory of Multimedia Learning (Figure 2).

FIGURE 2

FIGURE 2. Mayer’s cognitive theory of multimedia learning.

Microlearning

Microlearning is a learning strategy chosen to develop the instructional videos in this channel [22]. This strategy involves small, focused segments of learning that are each designed to meet a specific learning outcome. It allows for the content to be ‘chunked’ so as not to overload the learners’ cognition and allow for the ability to absorb, recall, retain, and practice. Each chunk should be around 5–7 min, which is ideal for motor skill teaching with higher engagement. There is a decline in engagement with an increased length of the video. The use of a video allows for pausing and replaying, acting as the foundation for self-paced learning. It also allows for ease of access across various devices, allowing for learning on demand to occur.

The advantages of microlearning are that it is performed in short time bursts, allowing learners to be alert; requires little effort from individual sessions; involves simple and/or narrow topics; is fun and engaging; allows for continuous updates; allows for multitasking; is casual and informal, and allows for learning anywhere and anytime [23].

However, the challenges of microlearning are apparent when there is learning that requires the acquisition of complex skills, processes, or behaviours. It is also apparent when relevant and contextual feedback on performance is needed, or there is a situation that prevents multi-tasking.

These platforms containing repositories of information and data generate a large volume of data, both structured and unstructured, have led to an explosion of digital data that can be explored and analyzed for future decisions and strategic planning [24]. TechAmerica Foundation defines big data as follows: “Big data is a term that describes large volumes of high velocity, complex and variable data that require advanced techniques and technologies to enable the capture, storage, distribution, management, and analysis of the information” [25]. Simply put, big data works on the principle that the more information is available about a subject, the greater the understanding and the more reliable will be the predictions about the future. This report explores the use of analytics data to direct and enhance the learning experience of the YouTube channel on Hand Surgery.

Aims

This study aims to describe, explore, and understand the global reach, use, and engagement patterns of a dedicated hand surgery education YouTube channel by analyzing viewer data generated by the platform.

Methods

This descriptive case study used an educational YouTube video channel (https://tinyurl.com/HANDSURGERYINTERNATIONAL) as a platform for the distribution of open educational resources in hand surgery. This method was chosen as it affords a holistic, in-depth examination of contemporary real-world challenges of surgical education. In this case study, the establishment and content creation are described for hand surgery training and its use as an open education resource for blended training.

The channel was set up in November 2008. It is populated with 262 instructional videos as an open educational resource and is contributed by six accredited surgeon content creators from Singapore and the United Kingdom. The videos are produced in accordance with multimedia instructional design principles by the content creators [26, 27]. The videos consist of surgical procedures, cadaveric dissections, demonstrations, and instructional lectures on Hand Surgery topics, inclusive of literature review and evidence-based practices.

The videos were also embedded as OER in an open online Moodle educational channel (https://handsurgery.activemoodle.com/). The content creators used the channel and the Moodle site as their source of OER for their flipped teaching activities like workshops, seminars, and lectures. All participants had free access to these OER and the channel for synchronous and asynchronous learning. The content creators conducted regular hand surgery teaching activities in Singapore, Cambodia, Bangladesh, Sri Lanka, India, Malaysia, United Kingdom, and the Netherlands. The videos were also posted to the social media platforms of Facebook (https://www.facebook.com/handsurgeryedu/) and Twitter (@handsurgeryedu) to augment learner reach.

Video capture was done using various high definition cameras, including dedicated professional cameras for laboratory-based recording to mobile device handheld cameras for surgical procedures. Postproduction was performed using PC computers with video editing suite Power Director, a prosumer video editing software by CyberLink, a multimedia software company. Didactic lectures were produced using the video production function of PowerPoint of the Microsoft Office productivity suite.

Although the videos though not formally peer-reviewed, were vetted by the group of six surgeons as the expert review panel. These videos are used by these surgeons and other surgical faculty as resource materials for various instructional workshops and lectures on Hand Surgery. The design and development of the videos have evolved with advancements in digital video technology and in accordance with microlearning principles.

Data Collection and Analysis

Data were extracted from the YouTube analytics data on June 21, 2020. The evaluation was based on the standard analytics data provided by YouTube. Variables included watch time, number of views, number of “likes” and “dislikes” for each video, and the number of channel “subscribers.” (defined as individuals who choose to receive notifications about new content in the channel. Viewer country and video topics were available for each view. General demographic data (i.e., viewer age and sex) were available in aggregate percentages for the channel. YouTube uses proprietary algorithms to ensure that view counts cannot be artificially inflated. Other variables include viewing habits, such as the number of subscribers, the total number of views, the watch time, average view duration, demographics, viewers’ locations, viewer’s comments, and more. The volume of information from the reports constituted as big data. It was used to provide more insight to help understand how the viewers use the videos and their perception to guide the design and development of future content.

Descriptive analyses were used to explore the data and trends in demographics and engagement on the channel. Continuous data (view duration) is presented as both medians and ranges, and means and standard deviations. Linear regression analysis was used to create a regression model to correlate impression click-through rate with the average view duration. Statistical analysis was performed on Microsoft Excel.

Results

Subscribers

Since 2008, there have been 6,521 subscribers to the channel, with most (4,663) of the new subscribers subscribing after 2018. There has been an increasing trend in the number of subscribers year on year. In 2018 and 2019, there were 1,060 and 1,984 new subscribers, respectively, and this number is set to increase in 2020, where from 1 January to June 21, 2020, there are already 1,619 new subscribers.

Views

There has been a total of 1,360,680 views, with a total view time of 35,033 h and an average view time of 1 min and 43 s. There is an increasing trend in the number of views over the years, where in 2018 and 2019, there were 197,946 and 312,482 views respectively, while there have been 239,081 views thus far in 2020, averaging 1,000 views per day.

The average percentage of the video viewed was 31.7%.

Average View Duration

The overall average view duration of the videos is consistently less than 2 min over the years. In 2018, the mean average view duration was 78.4 $\pm$ 68 s. The median duration was 61 (range 7–547) seconds. In 2019, the mean average view duration was 88.4 $\pm$ 74 s. The median duration was 72 (range 7–532) seconds. In 2020, the mean average view duration was 88.6 $\pm$ 72 s. The median duration 70 (range 8–568) seconds. The distribution can be seen in Figures 3–5.

FIGURE 3

FIGURE 3. Frequency of average view duration (in seconds in 2018).

FIGURE 4

FIGURE 4. Frequency of average view duration (in seconds in 2019).

FIGURE 5

FIGURE 5. Frequency of average view duration (in seconds in 2020).

Demographics

The demographics showed that 77% of viewers were males, 48.5% were 25–34 years of age, 21.5% were 35–44 years of age, and 18.9% were 18–24 years of age. The result is expected, as these videos were targeted to Hand Surgery trainees and junior surgical residents. The breakdown of views and watch times according to age groups can be seen in Figures 6, 7.

FIGURE 6

FIGURE 6. Hand surgery international YouTube channel views by age.

FIGURE 7

FIGURE 7. Hand surgery international YouTube channel watch time by age.

Most of the views (specified) came from the United States, accounting for 23.5% of the audience, 13.1% were from Malaysia, and 8.9% came from India. The United Kingdom made up 5.2% of the views. The breakdown of geographical distribution can be seen in Figure 8.

FIGURE 8

FIGURE 8. Geographical distribution of YouTube views.

Playback Location

The videos were viewed through mobile phones in 76.7%, while 16.0% were viewed through the computer. This can be seen in Figure 9. The average view duration in these platforms was 1 min and 29 s for mobile devices and 2 min and 3 s through the computer.

FIGURE 9

FIGURE 9. Playback devices used to view videos.

The majority (62.1%) utilized the Android platform, with iOS (18.1%) and Windows (13.5%) being the other two common platforms used.

Reach

Most of the traffic source (452,680 views, 33.3%) for the videos came from a YouTube search, while the second-highest traffic source (264,529 views, 19.4%) came from the suggested videos, and the third-highest traffic course (259,245 views, 19%) were from external links. These external links encompassed traffic from other websites. For example, 57.0% came from a Google search, 2% came from Facebook, and 1.2% came from Samsung Internet.

There has been a decreasing trend in the average view duration. In 2018, the average view duration was 1 min and 58 s; in 2019, it was 1 min and 30 s; and thus far, in 2020, it was 1 min and 15 s.

The most popular playlist was “flaps in hand surgery,” which accounted for 50.2% of the traffic to the channel. The second most popular playlist was “basic hand surgery workshop,” accounting for 21.9% of the traffic to the channel.

Impressions

There were 4,324,724 impressions with an average of a 7.32% click-through rate. The impact on impressions on watch time was only available after October 1, 2018. From October 1, 2018–June 21, 2020, there were 3,382,367 impressions, with YouTube recommending 15.3% of these impressions and 84.7% from viewers seeking the video content. There was a click-through rate of 7.2%, resulting in 244,701 views from impressions and a watch time of 7,520 h. The average view duration through impressions was 1 min and 50 s.

Impressions are created when a viewer comes across a video thumbnail of the channel, and this represents the frequency the thumbnails are shown on YouTube. It is a measure of the potential reach on YouTube and an opportunity to earn a view.

Impressions click-through rate shows what percentage of impressions on the video turned into views. This measures how often viewers watched a video after seeing an impression. While some views came from impressions, others (38%) came from sources with no thumbnail impression, such as views from the Moodle site and other social media sites like Facebook and Twitter, accounting for lowerclick-through rates.

Figure 10 demonstrates the impressions click-through rate and how they led to watch time.

FIGURE 10

FIGURE 10. Impressions click-through rate leading to watch time.

Linear regression analysis was performed to summarize a model to predict the value of impression click-through rate, with the x-variable being the average view duration (in seconds).

From 2018 to 2020, the average view duration was a significant indicator for impression click-through rate. As shown in Table 1, the p-value from the F-test and the regression model show that the model is significant. However, the adjusted R² demonstrates that there likely is a variation that is unable to be accounted for in this current model.

TABLE 1

TABLE 1. Linear regression analysis of impression click-through rate.

Likes

There was a total of 4,752 likes and a like-to-dislike ratio of 88.2%.

Shares

There was a total of 7,408 shares, with 2,065 (27.9%) via WhatsApp, 1,931 (26.1%) via copying to Clipboard, 1,606 (21.7%) through other social media means (apart from the text message, email, Facebook, and Twitter).

Audience Retention

The audience retention rates for the top 3 most-watched videos in the past 90 days were included–“Boxer’s Fracture with T Plate,” “VY Flap,”and “Trapeziectomy and APL Suspensioplasty”, and these rates can be seen in Figures 11–13 respectively. Absolute audience retention is the percentage of the number of views at a given time in the video as a percentage of the total number of the video views. Relative audience retention compares the audience retention of the videos to the average audience retention of all YouTube videos of similar lengths.

FIGURE 11

FIGURE 11. Audience retention rate for Boxer’s Fracture with T Plate. *The x-axis of the graph refers to the length of the video in percentage.

FIGURE 12

FIGURE 12. Audience Retention Rate for VY Flap. *The x-axis of the graph refers to the length of the video in percentage.

FIGURE 13

FIGURE 13. Audience Retention Rate for Trapeziectomy and APL Suspensioplasty. *The x-axis of the graph refers to the length of the video in percentage.

Re-Watches

Unique viewer was a tool introduced to YouTube Analytics only from August 1, 2017. The average number of re-watches is 35.7%. There is a declining trend in the average number of re-watches -- in 2017, the average number was 44.3%, in 2018, it was 40.1%, in 2019, it was 33.2%, and in 2020 (until June 21, 2020) was 26.8%.

Comments

There were 488 comments on the channel. The three videos with the top comments were “acute nerve repair” with 39 comments, “fractures of the terminal phalanx of the hand” with 22 comments, and “2nd metacarpal fracture ORIF with the plate” with 17 comments. Some of the comments were trivial or not specific to the video content; however, some comments on the video quality, included a request for subtitling. This was resolved with the use of speech recognition technology by YouTube to automatically create captions for narrated videos. Another comment was the dissatisfaction with the use of background music, which has been incorporated in the newer content with minimal music.

Discussion

The challenges of postgraduate surgical education have navigated learners toward an online learning environment, which allows for the flexibility to access educational content anytime, anywhere, and on-demand, for self-paced learning. Schroeder and colleagues demonstrated the importance of social technologies in providing a customisable learning environment for learners and autonomy and control over their learning [28] and enabling self-directed learning–described as “an initiative in which an individual takes, with or without the help of others, in finding out his or her own learning needs and goals, identifies the human and material resources for learning, decides and works out appropriate learning strategies, as well as evaluates the result of his or her learning.” YouTube affords itself as a self-directed learning environment as the content is easily shared and viewed universally [29]. It has also been reported as an effective platform to support procedural learning [30].

To the authors’ knowledge, this is the first study to analyze the viewing patterns of a dedicated YouTube channel on Hand Surgery. This study has shown that over 12 years, a wide reach globally was achieved with over 1.3 million views and more than 6,500 subscribers. The data showed a male preponderance of viewers in the 25–34 age group, reflecting the target audience of the subject matter covered in the channel. United States, and Asia accounted for the two largest subscribers. It has been shown that a vast majority (85%) of all view on YouTube is confined to just 3% of all channels on YouTube, of which educational videos account for less than 5% of all videos. There has been a significant reduction of views of videos, with 50% of all videos uploaded in 2016 had a total of only 89 views or less [31]. This study’s dedicated hand surgery channel had an average of 3,888 views per video, indicating a targeted group engagement. The concept of channel subscriptions promotes active learning where subscribers get notified whenever new content is added. The growth in subscribers over the last 2 years is due to the frequent hybrid hand surgery learning events (flipped workshops and instructional/review courses) conducted by the content creators. Increasing the channel’s reach and engagement requires the use of Search Engine Optimization (SEO) to attract traffic and grow a following. YouTube SEO involves optimizing the channel, by using video’s text in the form of transcripts, closed captions, and subtitles, including keywords in title, description, and tags. Thus, it is important for a full, accurate transcript, which currently is made available via YouTube speech recognition technology. The “thumbnail” (a random frame of an uploaded video that allows users to see a quick capture of its content) can also be optimized by creating a custom thumbnail that will encourage users to click on the video [32].

The average percentage of the video viewed (also known as retention rate) of 32% was low compared to an average of 50% of YouTube sites in general. This is partly due to the longer duration of the older videos as compared to the current. The short average duration of views seen in this study is in keeping with other medical education YouTube videos where the average view per minute is less than 4 min [33]. Also, ultra-short instructional videos of 1 min were shown to be effective in retention of skills after one view [34]. Optimal surgical instructional video duration is dependent upon the purpose of the video, the intended audience, the experience of the surgical faculty, and the type of procedures [35]. However, the relatively high click-through rates of 7.2% indicate a targeted audience for the channel - half of all channels and videos on YouTube have an impressions CTR that can range between 2 and 10%.

Thus, the title, descriptions, and thumbnail of the videos need to be carefully crafted to improve their viewing and retention rate [36]. There is a declining trend in the average number of re-watches, which could be due to the reduction in the frequency of uploads in the last year.

The channels features such as subscription counts, ranking, likes, and dislikes indicate the popularity and reception of the content. However, it has been shown that learners who watched educational videos on YouTube expressed positive sentiments as the platform affords online learners a positive self-directed learning environment. Despite an 88% like to dislike ratio, some of the dislikes (mainly due to the nature of the surgical content) were not related to the target audience. They were from the random public who may not understand the objectives of the channel.

Other studies have shown that online learners were generally positive about taking control of their learning via YouTube and were using the videos as supplemental resources for their formal learning. It has also been shown that learners who searched for educational videos were motivated individuals who wanted to have more control of their learning and more satisfied and confident that the knowledge gained from the resource would help them in the future [29].

The popularity of the various playlist corresponds to the nature of the content. The video on flaps in hand surgery was top. It represents basic surgical techniques for commonly used flap procedures in hand surgery (an information-intensive educational resources best served using video OER). It is also a useful resource for revising the steps before surgery. The other popular playlist resulted from their use as OER for flipped training workshops conducted by the content creators and other surgeon subscribers. Some of the generic factors to consider in the production of surgical instructional videos are a high-resolution production with the use of static images and animation diagrams. Short onscreen text and subtitling with background music and less background noise were also found to enhance the popularity of the videos. The fast pace of narration was also an important factor for a good instructional video [37].

This dedicated channel is a valuable repository for video OERs and can be used as a standalone or as part of a blended teaching activity in hand surgery. It is useful in surgical training in low resource countries [38]. Content creators can use these resources as legitimate scholarship assets and as part of the creators’ professional appraisal and advancement. The big data analysis of viewers’ habits can also guide the learning needs of surgical learners and therefore help in the design and development of the surgical curriculum and assessment.

The limitations of this study are the sole reliance on the data generated by the platforms, which lacked individual-level data, given that viewers cannot be tracked or identified (assume they represent the target population). There is indirect evidence that some of the viewers were non-surgical learners. The analytics algorithm is proprietary to YouTube and cannot be validated. YouTube Analytics data does not permit the analyses of the number of videos watched by individual subscribers or nonsubscribers for videos on the Hand Surgery International channel or other channels. As the topics covered in the index channel in this case study are not comprehensive for all topics in hand surgery, the inferences of the analysis are limited. Further, the conclusions will be difficult to generalize, especially in hand surgery, as online educational video platforms are not in abundance. One way of overcoming this is by triangulating this case study with other multiple-case design methods to confirm the validity of the findings.

It was beyond the scope of this study to systematically analyze the comments on the videos. This would require further qualitative research on a purposive sample of know users of the channel. It is crucial to understand that viewing a video does not correlate to learning, and this metric was outside the scope of this study. It would be useful if the YouTube platform allowed for in-video assessment via links in the video for external assessment. Further research has to be performed on the perceptions and acceptance of content creation on these social media sites as part of the scholastic achievement of academics and professional surgical practice. Views reach, and engagement of the bigdata in YouTube can be used as equivalent measures as citation is with traditional publications.

Conclusion

This case study conform to the theoretical framework of multimedia and micro-learning by online learners. It is recommended that educational videos as OER should be confined to 2 min, made compatible for the mobile device, and be optimized for sharing on social media. These can be used as resources for blended learning, allowing better utilization of time for deliberate practice in surgical training. YouTube has the potential to be an effective platform for self-directed and just in time learning among surgical learners. This study has shown the importance of content creators of surgical training to be cognisant of the potential of the big data analysis from YouTube to guide their pedagogical style and the curriculum development based on the collaborative process between contributors and consumers of the channel. With the current trends in remote learning, this study shows that it is possible for surgical trainers to build the resources for a digital learning platform to enable on-demand, self-paced and lifelong learning for surgical learners globally.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author Contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

1. Clement, J. YouTube - Statistics and facts (2020). Available from: https://www.statista.com/topics/2019/youtube/ (Accessed July 20, 2020).

2. Allgaier, J. Science and medicine on YouTube. In: J Hunsinger, MM Allen, and L Klastrup, editors. Second international handbook of internet research. Dordrecht:Springer Netherlands. p. 7–27. doi:10.1007/978-94-024-1555-1_1

CrossRef Full Text | Google Scholar

3. Rapp, AK, Healy, MG, Charlton, ME, Keith, JN, Rosenbaum, ME, and Kapadia, MR. YouTube is the most frequently used educational video source for surgical preparation. J Surg Educ (2016). 73:1072–6. doi:10.1016/j.jsurg.2016.04.024

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Mota, P, Carvalho, N, Carvalho-Dias, E, João Costa, M, Correia-Pinto, J, and Lima, E. Video-based surgical learning: improving trainee education and preparation for surgery. J Surg Educ (2018). 75:828–35. doi:10.1016/j.jsurg.2017.09.027

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Logan, R. Using YouTube in perioperative nursing education. AORN J (2012). 95:474–81. doi:10.1016/j.aorn.2012.01.023

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Schmidt, RS, Shi, LL, and Sethna, A. Use of streaming media (YouTube) as an educational tool for surgeons-A survey of AAFPRS members. JAMA Facial Plast Surg (2016). 18:230–1. doi:10.1001/jamafacial.2016.0007

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Barry, DS, Marzouk, F, Chulak-Oglu, K, Bennett, D, Tierney, P, and O'Keeffe, GW. Anatomy education for the YouTube generation. Anat Sci Educ (2016). 9:90–6. doi:10.1002/ase.1550

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Mayer, RE. Incorporating motivation into multimedia learning. Learn Instruction (2014). 29:171–3. doi:10.1016/j.learninstruc.2013.04.003

CrossRef Full Text | Google Scholar

9. Hanke, U. Generative learning. In: NM Seel, editor. Encyclopedia of the sciences of learning. Boston, MA: Springer US. p. 1356–8. doi:10.1007/978-1-4419-1428-6_171

CrossRef Full Text | Google Scholar

10. Bath-Hextall, F, Wharrad, H, and Leonardi-Bee, J. Teaching tools in Evidence Based Practice: evaluation of reusable learning objects (RLOs) for learning about Meta-analysis. BMC Med Educ (2011). 11:18. doi:10.1186/1472-6920-11-18

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Drozd, B, Couvillon, E, and Suarez, A. Medical YouTube videos and methods of evaluation: literature review. JMIR Med Educ (2018). 4:e3. doi:10.2196/mededu.8527

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Karic, B, Moino, V, Nolin, A, Andrews, A, and Brisson, P. Evaluation of surgical educational videos available for third year medical students. Med Educ Online (2020). 25:1714197. doi:10.1080/10872981.2020.1714197

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Rodriguez, HA, Young, MT, Jackson, HT, Oelschlager, BK, and Wright, AS. Viewer discretion advised: is YouTube a friend or foe in surgical education? Surg Endosc (2018). 32:1724–8. doi:10.1007/s00464-017-5853-x

PubMed Abstract | CrossRef Full Text | Google Scholar

14. de’Angelis, N, Gavriilidis, P, Martínez-Pérez, A, Genova, P, Notarnicola, M, Reitano, E, et al. Educational value of surgical videos on YouTube: quality assessment of laparoscopic appendectomy videos by senior surgeons vs. novice trainees. World J Emerg Surg WJES (2019). 14, 22. doi:10.1186/s13017-019-0241-6

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Parsons, BA, Blencowe, NS, Hollowood, AD, and Grant, JR. Surgical training: the impact of changes in curriculum and experience. J Surg Educ (2011). 68:44–51. doi:10.1016/j.jsurg.2010.08.004

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Sharma, SK, Palvia, SCJ, and Kumar, K. Changing the landscape of higher education: from standardized learning to customized learning. J Inf Technol Case Appl Res (2017). 19:75–80. doi:10.1080/15228053.2017.1345214

CrossRef Full Text | Google Scholar

17. Pugh, CM, Watson, A, Bell, RH, Brasel, KJ, Jackson, GP, Weber, SM, et al. Surgical education in the internet era. J Surg Res (2009). 156:177–82. doi:10.1016/j.jss.2009.03.021

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Azer, SA, AlGrain, HA, AlKhelaif, RA, and AlEshaiwi, SM. Evaluation of the educational value of YouTube videos about physical examination of the cardiovascular and respiratory systems. J Med Internet Res (2013). 15:e241. doi:10.2196/jmir.2728

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Friedl, R, Höppler, H, Ecard, K, Scholz, W, Hannekum, A, Öchsner, W, et al. Multimedia-driven teaching significantly improves students' performance when compared with a print medium. Ann Thorac Surg (2006). 81:1760–6. doi:10.1016/j.athoracsur.2005.09.048

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Abiri, A, Pensa, J, Tao, A, Ma, J, Juo, YY, Askari, SJ, et al. Multi-modal haptic feedback for grip force reduction in robotic surgery. Sci Rep (2019). 9:5016. doi:10.1038/s41598-019-40821-1

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Mayer, RE. How multimedia can improve learning and instruction. In: J Dunlosky, and KA Rawson, editors The Cambridge handbook of cognition and education. New York, NY, US: Cambridge University Press. p. 460–79. doi:10.1017/9781108235631.019

CrossRef Full Text | Google Scholar

22. Giurgiu, L. Microlearning an evolving elearning trend. Sci Bull (2017). 22:18–23. doi:10.1515/bsaft-2017-0003

CrossRef Full Text | Google Scholar

23. Jomah, O, Masoud, AK, Kishore, XP, and Aurelia, S. Micro learning: a modernized education system. Brain: Broad Res Artif Intell Neurosci 7:103–10. doi:10.5281/zenodo.1044220

CrossRef Full Text | Google Scholar

24. Gandomi, A, and Haider, M. Beyond the hype: big data concepts, methods, and analytics. Int J Inf Manage (2015). 35:137–44. doi:10.1016/j.ijinfomgt.2014.10.007

CrossRef Full Text | Google Scholar

25. Mills, S, Irakliotis, L, Co-Chair, A, Carlson, T, Lucas, S, Perlowitz, B, et al. Demystifying big data - a practical guide to transforming the business of government (2012). Available from: http://www.techamerica.org/Docs/fileManager.cfm?f=techamerica-bigdatareport-final.pdf (Accessed July 20, 2020).

26. Davis, G, and Norman, M. Principles of multimedia learning. Cent Teach Learn Wiley Educ Serv (2016). Available from: https://ctl.wiley.com/principles-of-multimedia-learning/ (Accessed July 20, 2020).

Google Scholar

27. Mayer, RE. Multimedia learning. 2nd ed. Cambridge, England: Cambridge University Press (2009). doi:10.1017/CBO9780511811678

CrossRef Full Text

28. Schroeder, A, Minocha, S, and Schneider, C. The strengths, weaknesses, opportunities and threats of using social software in higher and further education teaching and learning. J Comput Assist Learn (2010). 26:159–74. doi:10.1111/j.1365-2729.2010.00347.x

CrossRef Full Text | Google Scholar

29. Lee, CS, Osop, H, Goh, DHL, and Kelni, G. Making sense of comments on YouTube educational videos: a self-directed learning perspective. Oir (2017). 41:611–25. doi:10.1108/OIR-09-2016-0274

CrossRef Full Text | Google Scholar

30. Lee, DY, and Lehto, MR. User acceptance of YouTube for procedural learning: an extension of the technology acceptance model. Comput Educ (2013). 61:193–208. doi:10.1016/j.compedu.2012.10.001

CrossRef Full Text | Google Scholar

31. Bärtl, M. YouTube channels, uploads and views. Convergence (2018). 24:16–32. doi:10.1177/1354856517736979

CrossRef Full Text | Google Scholar

32. Lopezosa, C, Orduna-Malea, E, and Pérez-Montoro, M. Making video news visible: identifying the optimization strategies of the cybermedia on YouTube using web metrics. Journalism Pract (2020). 14:465–82. doi:10.1080/17512786.2019.1628657

CrossRef Full Text | Google Scholar

33. Tackett, S, Slinn, K, Marshall, T, Gaglani, S, Waldman, V, and Desai, R. Medical education videos for the world: an analysis of viewing patterns for a YouTube channel. Acad Med (2018). 93:1150–6. doi:10.1097/ACM.0000000000002118

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Bobrow, BJ, Vadeboncoeur, TF, Spaite, DW, Potts, J, Denninghoff, K, Chikani, V, et al. The effectiveness of ultrabrief and brief educational videos for training lay responders in hands-only cardiopulmonary resuscitation. Circ Cardiovasc Qual Outcomes (2011). 4:220–6. doi:10.1161/CIRCOUTCOMES.110.959353

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Yee, A, Padovano, WM, Fox, IK, Hill, EJR, Rowe, AG, Brunt, LM, et al. Video-based learning in surgery: establishing surgeon engagement and utilization of variable-duration videos. Ann Surg (2020). 272:1012–9. doi:10.1097/SLA.0000000000003306

PubMed Abstract | CrossRef Full Text | Google Scholar

36.Tips for using impressions & CTR data - YouTube Help. Available from: https://support.google.com/youtube/answer/7628154?hl=en (Accessed July 20, 2020).

37. Nasir, AR, and Bargstädt, HJ. An approach to develop video tutorials for construction tasks. Proced Eng (2017). 196:1088–97. doi:10.1016/j.proeng.2017.08.066

CrossRef Full Text | Google Scholar

38. Cheok, S, Dong, C, Lam, WL, Gollogly, J, and Rajaratnam, V. A model for surgical volunteerism: a qualitative study based in Cambodia. Trop Doct (2020). 50:53–7. doi:10.1177/0049475519884442

PubMed Abstract | CrossRef Full Text | Google Scholar