Natural Stings: Selling Distrust About Vaccines on Brazilian YouTube

1. Introduction

The World Health Organization considered vaccine hesitation as one of the greatest threats to global health in 2019 (World Health Organization, 2019b). With the decrease in immunization coverage, 170 countries registered cases of measles, increasing 300% when compared to the first three months of 2018 (World Health Organization, 2019a). In this scenario, Brazil was the sixth country by reported cases of measles in 2019 (Castelvecchi et al., 2019), despite the National Immunization Program, one of the strongest Brazilian public health policy programs (Domingues et al., 2012), offering 20 different types of vaccines free of charge to the population (G1 DF, 2020). Researchers associated the decrease in immunization to a strengthening of the anti-vaccination movements (Dubé et al., 2015; Benecke and DeYoung, 2019), suggesting that social media is currently the main spreader of this position (Stecula et al., 2020).

In the last two decades, several researchers studied vaccine misinformation and disinformation (M&D) on different online platforms and sites, such as Facebook (Faasse et al., 2016; Orr et al., 2016; Hoffman et al., 2019), YouTube (Keelan et al., 2007; Ache and Wallace, 2008; Briones et al., 2012; Cambra et al., 2016; Covolo et al., 2017), among others (Nasir, 2000; Wolfe et al., 2002; Zimmerman et al., 2005; Kata, 2010; Bean, 2011). According to them, the amount of vaccine M&D on sites and social media platforms have varied throughout the time (Ache and Wallace, 2008; Briones et al., 2012). Despite that, the online presence of anti-vaccination movements seems to increase in recent years (Donzelli et al., 2018; Johnson et al., 2020). These communities use strategies to reach a larger audience—personal narratives (Duchsherer et al., 2020), anti-vaccination advertisements (Jamison et al., 2020)—and create workarounds to avoid being flagged as “harmful content” on social media—e.g., using lexical variations and emphasizing concepts like “informed consent,” “health freedom,” and “pro-choice” (Kata, 2012; Koltai, 2020; Szeto et al., 2020).

YouTube increasingly becomes one of the most popular social media services, reaching over two billion users each month (YouTube About, 2020). YouTube's recommendation system is responsible for 70% of total viewing time, and drives the promotion of certain content within the site (Popken, 2018). YouTube's reach is a reason for concern, since its recommendation system is identified as a promoter of misinformation and extreme content (Mozilla, 2019; Avaaz, 2020). There is still a gap in the research literature on vaccine M&D in other languages than English and when investigating this topic, great part of the researchers draw attention to other platforms, such as Facebook and Twitter. As suggested by Allgaier (2018), investigating YouTube empirically is still challenging because of the numerous approaches and problematic aspects associated with it, reinforcing the importance of research aimed at this online environment.

After a series of complaints from academics and journalists (Geurkink, 2019), YouTube stated the implementation of measures to reduce recommendations of harmful misinformation and to remove advertisements from anti-vaccination videos (Ingram, 2019). Such claims can not be confirmed without access to its data. Also, these measures are even more complex when considering content in languages other than English and countries without proper regulation. For instance, YouTube has great influence in the spread of anti-vaccination content in Portuguese: almost 90% of the videos analyzed in Avaaz and SBIm (2019) presented some misinformation about vaccines, reaching 7.4 mi of visualizations in the last three years. These are concerning results, since 57% of the interviewed subjects that did not vaccinate themselves or their children declared misinformation about vaccines as a reason (Avaaz and SBIm, 2019).

In this paper, we investigated M&D about vaccines using a case study approach. We used a sample obtained from the terms “vacina+autismo” (vaccine+autism, in English), to understand how M&D about vaccines circulate on YouTube in Portuguese, and who are the channels creating and disseminating this kind of content. These terms were used due to the prevalence of the false link between vaccines and autism (Suelzer et al., 2019), a disinformation spread first in 1998 with the publication of a fraudulent paper (Deer, 2011) and still used as an argument by the anti-vaccination movements (Venkatraman et al., 2015; van Schalkwyk, 2019).

Despite YouTube's statements on their fight against misinformation (O'Donovan, 2019), we verified that channels already identified as creators of harmful M&D (Avaaz and SBIm, 2019) continue to offer questionable videos to the public, with many of them still being part of the YouTube Partner Program. In our sample, we found a community of content creators that collaborate to promote alternative health services meanwhile spread M&D about vaccines. These channels have financial gains selling courses, books, and alternative treatments, requesting donations through fundraising platforms and deposits in bank accounts, and even from large companies, through advertisements on YouTube.

2. Materials and Methods

2.1. Acquiring and Pre-processing the Data

We used YouTube Data Tools (Rieder, 2019) to obtain the data used in this study. The tool was set up using the option “Video Network,” which creates a network of relations between videos via YouTube's “related videos” feature: YouTube defines “related videos” from the relatedToVideoId parameter, which contains a list of videos related to a given video (YouTube, 2020). We selected the following parameters on YouTube Data Tools:

• Search query: “vacina+autismo” (in English, “vaccine+autism”)

• Iterations: 1

• Rank by: relevance

• Crawl depth: 2

The data was obtained in triplicate on February 10, 2020, between 19:00 and 21:00 UTC. This way, we obtained 1,714, 2,135, and 2,286 videos in each extraction. Then, we combined the three datasets and used Gephi (Bastian et al., 2009) to obtain specific videos: containing the word vacina—vaccine—in their title; connected to the network, with a network degree higher than one—i.e., connected to the network by more than one related video; with more than 10,000 views. We used the following queries on Gephi:

Giant Component → Degree Range: ≥ 2 → Range (viewcount): → 10000

These processes reduced the data to 191 videos. From these videos, we excluded the irrelevant ones—information on animal vaccination, technical material addressed to health workers, advertisements and videos on the Vaccine Revolt, a historical moment in Brazil. These videos were watched and analyzed completely, and were considered as irrelevant due to their content; for example, videos on the Vaccine Revolt were created by official institutions or science communication channels that approach the event from a historical perspective. Then we reached the initial sample of our study, consisting of 158 videos.

We performed the classification on the initial sample between February 11 and 18, 2020, using a protocol of vaccine misinformation and disinformation (M&D) developed for this research. All videos were watched completely; when vaccine M&D was identified, the content was manually classified by an author (DFTM). If any of the videos created uncertainty, a second author (LG) would analyze the material and both would discuss to reach consensus. Several videos had more than one category of M&D about vaccines. We classified M&D categories in each video, and our classification is presented in the Supplementary Material (de Siqueira, 2020) in order of appearance. We also identified and documented the brands advertising on each video manually throughout screenshots, available—along with the Supplementary Material—on a GitHub repository (de Siqueira, 2020). We watched the videos using the Tor browser (The Tor Project, Inc, 2020), to minimize YouTube's algorithm system personalization.

We also created a secondary sample visiting the channels containing M&D videos from the initial sample between March 3 and 11, 2020. The videos selected contained the word “vacina,” or related to vacina—e.g., VACINAS: Você acredita Nelas? (VACCINES: Do you believe in them?)—in the title. These videos were classified between March 12 and 20, 2020, using the same protocol from the initial sample. Therefore, our sample has 52 videos containing M&D about vaccines in total—23 from the initial sample and 29 from the second sample.

2.2. Classifying the Videos

To specify what videos on our sample contain misinformation or disinformation, we labeled them according to six main categories created for this study. Each category consists of the main M&D spread about vaccines:

1. Safety: vaccines cause diseases (A); vaccinated people transmit the disease (B); vaccines cause autism (C); vaccines cause severe collateral effects (D); vaccines contain dangerous ingredients (E); it is safer to contract the disease than vaccinate (F); vaccines can impair the immune system (G); overload on vaccines and its consequences (H); alternative vaccination schedules are safer (I)

2. Effectivity: vaccines do not work (J); vaccines are not responsible for the decrease in diseases (K)

3. Alternative health: promotion of alternatives to vaccination, mainly naturopathic and wellness services (L)

4. Morality: association between HPV vaccine and promiscuity, and/or religious issues (M)

5. Conspiracy theories: narratives about powerful institutions or actors with nefarious intentions and secret plans (N)

6. Other: self-direction—freedom of choice, independent research (O); claim that people that know the “truth” do not vaccinate (P); emotional appeal (Q)

These categories are based on myths about vaccination most heard by Brazilian physicians, most spread M&D about vaccines in Brazil (Avaaz and SBIm, 2019), and in research analyzing the content of sites or videos about vaccination on the last two decades (Nasir, 2000; Wolfe et al., 2002; Zimmerman et al., 2005; Kata, 2010; Bean, 2011; Ward et al., 2015; Ekram et al., 2019; Yiannakoulias et al., 2019). Self-direction, as presented in the category Other, is based on Koltai's research on the anti-vaccination movement in the United States (Koltai, 2020).

2.3. License and Reusability

The algorithms and functions implemented in this study were written in Python (van Rossum and Drake, 2009), using the packages Numpy (Oliphant, 2006; van der Walt et al., 2011; Harris et al., 2020), Scipy (SciPy 1.0 Contributors et al., 2020), and Matplotlib (Hunter, 2007). All code and data published with this paper is available under the BSD 3-clause license, and all figures generated in this paper are available under the CC-BY 4.0 license.

3. Results

Our initial sample contained 158 videos. We classified 23 videos from this sample with at least one misinformation or disinformation (M&D) about vaccines. The main categories identified were Safety—dangerous ingredients in vaccines (E)—, Alternative health—the promotion of products and therapies as vaccine alternatives (L)—, and Other—the defense of self-direction (O). These specific M&D examples, E, L, and O, were mentioned in at least 10 videos. On the second sample, the main categories were Safety—vaccines cause diseases (A), severe collateral effects (D), dangerous ingredients (E)—, Alternative health—promotion of alternatives (L)—, Conspiracy theories—vaccines are part of conspiracies (N)—, and Other—defense of self-direction (O). These M&D examples were also mentioned in 10 or more videos. In total, the main examples of M&D in both samples were E (appearing in 28 videos), O (25 videos), L (22 videos), A (18 videos), N (17 videos), and D (14 videos).

To characterize how the amount of M&D in a video is related to its duration and the engagement it received, we defined an engagement metric, ε, based on the number of views, comments, likes, and dislikes in the videos containing M&D in both samples:

$\begin{array}{l}\epsilon =\frac{N_{likes}+N_{dislikes}+N_{comments}}{N_{views}}*100\end{array}$

We could apply the engagement metric in 43 of the 52 videos, since the other ones do not make their numbers of likes, dislikes, or comments available. The largest part of our sample that contains M&D—38 videos out of 43—has <40 min and 20% of engagement (Figure 1).

FIGURE 1

Figure 1. (A) Engagement metrics—views, comments, likes, and dislikes—from videos containing misinformation and disinformation (M&D) in our sample. (B) Quantity of M&D in each video, according to its duration and engagement. Here we represent only 43 videos, since the other ones do not make their numbers of likes, dislikes, or comments available. The largest part of our M&D sample—38 videos out of 43—has <40 min and 20% of engagement. Y axes in logarithmic scale. Colormap: YlOrRd.

Considering both samples, 32 videos referred to vaccines in general. Some discuss specific vaccines: 14 of them mentioned flu vaccines; 12 cited the yellow fever vaccine; seven discussed the HPV vaccine; two referred to the measles vaccine and yet meningitis B, varicella, rubella, smallpox, diphtheria, polio, whooping cough, and tuberculosis vaccines were each mentioned in one video. An interesting find is the reference to a cancer vaccine in four videos of the sample. According to the content producers, vitamin D and sun exposure would be the “real” cancer vaccine, a claim that appears in videos that promote alternative services in regard to vaccination.

We identified 39 brands advertising on at least one M&D video in our sample through YouTube: 5econds, 7 Springs Orthopedics, Adler Pharma, Andreas Grosz, Banza, Baumdick, Boiron, Bondic, Buscopan (Boehringer Ingelheim), Christen in Not, Datrium, Digital Dream Lifestyle, DG Achieve, Erie Metal Roofs, Eucerin (Beiersdorf), Fiat, Happn, Health and Wellness Tools, Incredible India, Japan Gov, Kia, Leap4Freedom, Lecturio, Lume Deodorant, Mindvalley, Mobil, OPPO, Patrick van Diemen, Philips, PragerU, RapidfFN, SDI Broker, SEAT, Spotify, Ticketmaster, The Online Traveler, Unichamp, Vileda, and Weiterdenker. Besides YouTube advertisements, the channels in our sample profit in other ways: they sell courses, books, alternative treatments, ask for donations through fundraising platforms or via deposits directly into bank accounts.

Another relevant aspect is the number of verified channels in the sample. Verified channels are official channels of a creator, artist, company, or public figure. Channel verification helps distinguishing official channels from other ones with similar names (YouTube Help, 2020). Eight channels, from 20 in our sample, are verified (Table 1): two are associated with broadcast TV programs, while the other six are self-made amateur videos.

TABLE 1

Table 1. Verified channels spreading M&D about vaccines and main examples of M&D.

The main M&D examples spread by the eight verified channels are the promotion of alternatives to vaccination (L)—present in seven channels—, the defense of self-direction (O)—cited in five channels—, and the allegation of dangerous ingredients in vaccines (E)—mentioned in four channels. The channels are also subscribed to different YouTube categories, indicating that M&D about vaccines can appear in several content spaces; for example, the eight verified channels have categories ranging from “Education” to “People & Blogs” (Table 1).

There is a collaboration between channels that promote alternative health services. From 20 channels spreading M&D about vaccines, 11 mentioned Lair Ribeiro (Dr. Lair Ribeiro Oficial), a cardiologist and nutrologist that promotes alternative therapies, diets, and pseudoscience—homeopathy, detox, law of attraction, quantum medicine—in his videos and talks. The collaboration occurs through the reproduction of videos from “associate” channels or via endorsement of content creators and their courses (Figure 2). Besides that, the channels promote other professionals that support alternative therapies or other content creators that endorse M&D about vaccines.

FIGURE 2

Figure 2. Collaboration between channels that promote alternative health services and M&D about vaccines. Eleven channels spreading M&D in our sample endorse Dr. Lair Ribeiro Oficial. There are collaborations from other channels as well; for example, Minha Saúde—Curas Naturais and nutrição Alimentos & Cia endorse each other.

One strategy used by six channels is to ask testimonials from the audience, aiming to demonstrate the effectiveness of alternative therapies or courses they promote. Besides sending testimonials, the public has access to other services using social media platforms such as WhatsApp or Telegram. Some channels send daily “health tips,” offer discounts to products and claim that the use of these services are necessary to ensure that the public will receive new contents. Five of 20 channels make these communication services available, and one channel maintains 10 WhatsApp groups.

We also analyzed the upload date from the videos containing M&D in our sample to verify if YouTube could be in the process of removing videos from the sample that could have been recently uploaded. Twenty three videos were uploaded in 2018; seven were uploaded in 2017, and six were uploaded in 2019. The sample also had videos uploaded in 2016 (two), 2015 (five), 2014 (three), 2011 (two), and 2010 (four). From 23 videos uploaded in 2018, eight still had advertisements associated with it. While we extracted the data from our sample, the videos uploaded in 2018 accumulated 445,519 views.

4. Discussion

The belief that vaccines contain dangerous ingredients (E), the “toxin gambit,” persists between the anti-vaccination movements (Kata, 2012). The videos mention specific ingredients such as aluminum, mercury, formaldehyde, thimerosal, nagalase, aborted fetal tissues, poison, toxins, and chemicals. This belief is associated with two other M&D from the Safety category—vaccines cause diseases (A) and vaccines cause severe collateral effects (D). The narrative of unsafe vaccines is strengthened by the popularity of conspiracy theories (N), sowing doubt about public health organizations, physicians, scientists, universities, and mainstream media.

Conspiracy theories are attempts to explain events and circumstances from secret plots by powerful and malevolent actors (Douglas et al., 2019). The spread of conspiracy theories is a matter of concern, since exposition to conspiracy theories may have negative consequences, such as decreasing science acceptance (van der Linden, 2015), reduction in intention to engage in politics (Jolley and Douglas, 2014b) and, in the specific case of anti-vaccine conspiracy theories, can affect vaccination intentions (Jolley and Douglas, 2014a) and the pursuit of alternative vaccination schedules (Callaghan et al., 2019). The issue of mistrust—of vaccines, vaccination providers, or policy makers—is considered a driver for vaccine hesitation (Brown et al., 2018). Thus, the spread of distrust acts as a strategy to promoters of alternative therapies: in doing so, they affirm themselves as trusted sources for the audience and create potential consumers to the courses, books, and therapies they sell. The association between vaccine opposition and alternative health services has been made several times before (Ernst, 2001; Busse et al., 2008; Kata, 2010); for instance, Caulfield et al. (2017) analyzed 330 naturopath websites to understand the role of complementary and alternative medicine providers on the vaccine hesitation, and found that 40 of them presented anti-vaccination rhetoric, while 26 promoted vaccine alternatives.

The channels offered alternatives for vaccines and for treatments of health problems in a narrative of return to nature, as seen before by Kata (2012). Expressions like “God's pharmacy” and “disease's industry” are repeated to denominate the opposition between the right option—alternative products and therapies—and the “unnatural,” wrong one—vaccines, drugs, and mainstream medicine. “Disease's industry” is a reference to one of the most reiterated arguments of anti-vaccination conspiracy theories, where governments, scientists and physicians would be involved in a conspiracy with pharmaceutical companies to cause suffering and to profit (Jolley and Douglas, 2014a). Therefore, the advocates for alternative health services present themselves simultaneously as brave antagonists and victims persecuted for their resistance to a “corrupt system” (Lewandowsky and Cook, 2020).

A common strategy used by the channels is the production of free materials such as recorded talks, courses and e-books. The offering of free content helps to reinforce the idea of a selfless mission where the creators have no financial interests and is also a tactic to attract and retain an audience for the channels. Two of the content creators, Ribeiro and Bussade—an orthomolecular physician and nutrologist that promotes the chemical compound known as Miracle Mineral Solution and vitamins as treatments for autism—affirm that they do not practice medicine, but continue their studies and produce content because they are motivated by the mission of helping people to find good recommendations of natural health and cures for their sufferings. This mission is reinforced by their claims of persecution, an argument present in vaccination opposed spaces (Avaaz and SBIm, 2019; Smith and Graham, 2019; Koltai, 2020). Ribeiro, for example, criticizes the unpreparedness of traditional physicians and accuses the pharmaceutical industry. Kata (2012) describes this trope as the “Galileo gambit,” an argument that invokes the attacks from the scientific orthodoxy of the past against scientists such as Galileo Galilei—the claim is that the currently persecuted ideas will eventually be accepted as truth.

The alleged persecution cited by the content creators in our sample is also associated with the recent changes in social media policies to contain the spread of M&D. The content creators adopt workarounds to the moderation practices, as seen on anti-vaccine communities on Facebook (Koltai, 2020), for instance. One of the workarounds of the alternative health channels is using other social media platforms, such as Telegram or WhatsApp, to ensure the content circulation and the perception of community in and out of YouTube. WhatsApp is one of the most popular information resources for Brazilians (Deloitte, 2019), and it has favored the spread of health and political misinformation in the last years (Smallman, 2018; Evangelista and Bruno, 2019; Resende et al., 2019). In addition to enabling constant interactions with community members, these WhatsApp and Telegram groups encourage the sharing of audience testimonials, which are used in new videos aiming to demonstrate the effectiveness of alternative therapies and courses promoted by the channels. Testimonials or anecdotes are also an important tactic used by anti-vaccination communities (Davies et al., 2002; Zimmerman et al., 2005; Kata, 2010; Shelby and Ernst, 2013; Ward et al., 2015; Duchsherer et al., 2020; Rallo Shimizu, 2020), possibly impacting the perception of risk about vaccines (Betsch et al., 2011). Apart from the sample's six channels asking for testimonials from the audience, another one used a dramatic testimonial of a young girl to alarm the viewers against the HPV vaccine.

According to the vaccine opposed conspiracy narrative, people need to exercise their self-direction—as in freedom of choice and independent research—to protect themselves and their families from malevolent and powerful actors. Freedom, choice and individuality are values associated with anti-vaccination beliefs (Moran et al., 2016). The emphasis on freedom of choice is a strategy used by vaccine opponents to rebrand the anti-vaccination movement and to avoid being censored by platforms (Koltai, 2020). The need for independent research about vaccines—as in research for yourself; the ability to do research for oneself and to ask questions is cited as a condition to freedom (Koltai, 2020)—is a recurrent argument among vaccine opposed actors to advocate for more informed choices (Hoffman et al., 2019; Duchsherer et al., 2020). However, the correct way to conduct independent research is already set by the alternative health community narratives—since they are part of a conspiracy plan, mainstream media, traditional physicians, and public health organizations cannot be trusted. Meanwhile, the constant collaboration between channels that promote alternative health services affirm content creators as reliable sources for their viewers, strengthening a sense of community and belonging (Koltai, 2020; Rallo Shimizu, 2020) that favors the creation of a loyal audience. Duchsherer et al. (2020) analyzed parent testimonials from the anti-vaccination documentary Vaxxed and found an association between community building and the division of people and institutions in two groups: trustworthy—community members such as parents who do not vaccinate, like-minded online groups, and alternative medical professionals—and adversaries—governmental agencies and health providers that endorse vaccination protocols.

The collaboration into the vaccine M&D network resembles the behavior of the Alternative Influence Network (AIN), as described by Lewis (2018). The AIN is a content creators network on YouTube that is driven by a “set of shared ideas about progressive politics and social justice” and that uses, among other things, referrals, and guest appearances of different creators in videos as an strategy to build an audience (Lewis, 2018). Beyond that, the AIN spreads distrust about the mainstream media to present itself as an “alternative” media system, claims an alleged persecution because of their beliefs, encourages the audience to do “their own research,” and uses different strategies to monetize content. According to Tripodi (2018), besides enabling the reach of new and larger audiences, the network strategy helps to reaffirm the same narratives and positions between channels.

An important aspect of the spreading dynamics of M&D about vaccines is the adoption of different strategies to profit. Although researchers already identified advertisements running on climate misinformation videos (Allgaier, 2019; Avaaz, 2020) and on videos disseminating M&D about vaccines (Avaaz and SBIm, 2019), and despite YouTube's statements about measures to fight harmful misinformation on the platform (O'Donovan, 2019), we found that monetization of these kind of content still persists. We identified 39 brands advertising on 13 videos of our M&D sample. Besides global brands such as Mobil, Kia, Fiat, Philips, Spotify, Eucerin (Beiersdorf), and Buscopan (Boehringer Ingelheim), we found advertisements from the governments of India and Japan. We also highlight the presence of advertisements from companies of alternative health and wellness products—Boiron, Lume Deodorant, Adler Pharma, and Health and Wellness Tools. It is possible that the brands could be unaware that they are helping to fund channels that spread M&D about vaccines, although alternative health advertisers are potentially reaching the audience they want to reach.

Although the YouTube Partner Program is an important source of income, it is not the only funding option for content creators (Alexander, 2019). They maintain profiles on fundraising platforms or sell merchandise as well, and selling products and courses is a regular activity, particularly between alternative health channels (Avaaz and SBIm, 2019). For example, the host of the channel nutrição Alimentos & Cia encourages the audience to buy alternative health courses in a large number of their videos. To validate these courses, he stresses the adoption of alternative health practices by the Brazilian Unified Health System.

Despite YouTube's statements on their fight against misinformation, M&D about vaccines continue to be disseminated in videos in Portuguese, reaching a large audience. Even when channels are identified as creators of harmful misinformation, their videos continue to be available to the public and can be disseminated in other platforms, such as WhatsApp, as we found out in this research. For example, five channels from our sample— nutrição Alimentos & Cia, O Melhor Para Você, U MIÓ QUE TÁ TENO, Dr. Lair Ribeiro Oficial, and Ana Paula Palagar —were already identified by Avaaz and SBIm (2019) and all of them are part of the collaboration's network described in this study (Figure 2). We found that M&D about vaccines are associated with alternative health channels that spread distrust about traditional institutions—public health organizations, physicians, scientists, universities, and mainstream media—to promote and profit with alternative health services. YouTube must adopt transparent approaches to counter M&D and ensure that content creators follow the user policies. The platform also needs to guarantee that M&D will not be financially stimulated through the YouTube Partner Program.

Special attention should be given for languages other than English, since YouTube's current policies to fight harmful content do not seem to be applied to these languages. For that, YouTube needs to have a qualified team of human content moderators for different countries and languages. Our research suggests that their automated filters are not capable of identifying certain types of harmful content in Portuguese. Kata (2012), Koltai (2020), and Szeto et al. (2020), for instance, identified content creators that spread misinformation using strategies such as the replacement of “problematic terms,” to prevent the platforms to recognize content that does not follow the platform policy, an extra challenge for automatic systems. In addition to the urgence to ensure that videos with M&D do not be incentivized and rewarded, advertisers need to track the content that their publicity revenue is funding and need to ask for effective mechanisms to exclude their ads from harmful content. Other specific responses are frequently suggested (e.g., Caplan et al., 2018; Ghosh and Scott, 2018; Lewis, 2018; Nadler et al., 2018; Mozilla, 2019; Avaaz, 2020), and should be considered by YouTube, since the collaboration with researchers could lead to better policies and mechanisms that ensure that effective policies are implemented and respected.

Data Availability Statement

The data generated for this study can be found on GitHub: https://git.io/JJfVK.

Ethics Statement

Written informed consent was not obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author Contributions

DFTM, AFS, and LG conceived the study. DFTM and AFS collected the data and wrote the manuscript. All authors read and approved the final version of the manuscript.

Funding

This research was funded in part by the Gordon and Betty Moore Foundation through Grant GBMF3834 and by the Alfred P. Sloan Foundation through Grant 2013-10-27 to the University of California, Berkeley.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors thank Leandro Tessler (IFGW—Unicamp, Brazil) for reading a preliminary version of this manuscript. LG thanks also the support of Laboratório de Tecnologias e Transformações Sociais (LABTTS—Unicamp, Brazil).

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcomm.2020.577941/full#supplementary-material

References

Ache, K. A., and Wallace, L. S. (2008). Human papillomavirus vaccination coverage on youtube. Am. J. Prevent. Med. 35, 389–392. doi: 10.1016/j.amepre.2008.06.029

CrossRef Full Text | Google Scholar

Alexander, J. (2019). Youtube Looks to Demonetization as Punishment for Major Creators, But It Doesn't work. The Verge.

Allgaier, J. (2018). “Science and medicine on YouTube,” in Second International Handbook of Internet Research, eds J. Hunsinger, L. Klastrup, and M. M. Allen (Wiesbaden: Springer). doi: 10.1007/978-94-024-1202-4_1-1

CrossRef Full Text | Google Scholar

Allgaier, J. (2019). Science and environmental communication on youtube: strategically distorted communications in online videos on climate change and climate engineering. Front. Commun. 4:36. doi: 10.3389/fcomm.2019.00036

CrossRef Full Text | Google Scholar

Avaaz (2020). Why is YouTube Broadcasting Climate Misinformation to Millions? Avaaz.

Avaaz and SBIm (2019). As Fake News estão nos Deixando Doentes? Avaaz and SBIm.

Bastian, M., Heymann, S., and Jacomy, M. (2009). “Gephi: An open source software for exploring and manipulating networks,” in Annals of International AAAI Conference on Weblogs and Social Media (San Jose, CA). Available online at: https://aaai.org/Conferences/ICWSM/icwsm09.php

Google Scholar

Bean, S. J. (2011). Emerging and continuing trends in vaccine opposition website content. Vaccine 29, 1874–1880. doi: 10.1016/j.vaccine.2011.01.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Benecke, O., and DeYoung, S. E. (2019). Anti-vaccine decision-making and measles resurgence in the united states. Glob. Pediatr. Health 6:2333794X19862949. doi: 10.1177/2333794X19862949

PubMed Abstract | CrossRef Full Text | Google Scholar

Betsch, C., Ulshöfer, C., Renkewitz, F., and Betsch, T. (2011). The influence of narrative v. statistical information on perceiving vaccination risks. Med. Decis. Mak. 31, 742–753. doi: 10.1177/0272989X11400419

PubMed Abstract | CrossRef Full Text | Google Scholar

Briones, R., Nan, X., Madden, K., and Waks, L. (2012). When vaccines go viral: an analysis of hpv vaccine coverage on youtube. Health Commun. 27, 478–485. doi: 10.1080/10410236.2011.610258

PubMed Abstract | CrossRef Full Text | Google Scholar

Brown, A. L., Sperandio, M., Turssi, C. P., Leite, R. M. A., Berton, V. F., Succi, R. M., et al. (2018). Vaccine confidence and hesitancy in Brazil. Cadernos de Saúde Pública 34, 1–12. doi: 10.1590/0102-311x00011618

PubMed Abstract | CrossRef Full Text | Google Scholar

Busse, J. W., Wilson, K., and Campbell, J. B. (2008). Attitudes towards vaccination among chiropractic and naturopathic students. Vaccine 26, 6237–6243. doi: 10.1016/j.vaccine.2008.07.020

PubMed Abstract | CrossRef Full Text | Google Scholar

Callaghan, T., Motta, M., Sylvester, S., Lunz Trujillo, K., and Blackburn, C. C. (2019). Parent psychology and the decision to delay childhood vaccination. Soc. Sci. Med. 238:112407. doi: 10.1016/j.socscimed.2019.112407

PubMed Abstract | CrossRef Full Text | Google Scholar

Cambra, U. C., Díaz, V. C., and Herrero, S. G. (2016). Vacunas y anti vacunas en la red social youtube. Opción 9, 447–465. Available online at: https://dialnet.unirioja.es/servlet/articulo?codigo=5891177

Google Scholar

Caplan, R., Donovan, J., Hanson, L., and Matthews, J. (2018). Algorithmic Accountability: A Primer. Data & Society.

Castelvecchi, D., Cyranoski, D., Gibney, E., Ledford, H., Maxmen, A., Morello, L., et al. (2019). The science news events that shaped 2019. Nature 576, 350–353. doi: 10.1038/d41586-019-03838-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Caulfield, T., Marcon, A. R., and Murdoch, B. (2017). Injecting doubt: responding to the naturopathic anti-vaccination rhetoric. J. Law Biosci. 4, 229–249. doi: 10.1093/jlb/lsx017

CrossRef Full Text | Google Scholar

Covolo, L., Ceretti, E., Passeri, C., Boletti, M., and Gelatti, U. (2017). What arguments on vaccinations run through youtube videos in Italy? A content analysis. Hum. Vaccines Immunother. 13, 1693–1699. doi: 10.1080/21645515.2017.1306159

PubMed Abstract | CrossRef Full Text | Google Scholar

Davies, P., Chapman, S., and Leask, J. (2002). Antivaccination activists on the World Wide Web. Arch. Dis. Childhood 87, 22–25. doi: 10.1136/adc.87.1.22

PubMed Abstract | CrossRef Full Text | Google Scholar

de Siqueira, A. F. (2020). Natural Stings: Alternative Health Services Selling Distrust About Vaccines on Youtube - Supplementary Material. Available online at: https://github.com/alexdesiqueira/publications/tree/master/2020/yt_recommendation

Google Scholar

Deer, B. (2011). How the case against the MMR vaccine was fixed. BMJ 342. doi: 10.1136/bmj.c5347

PubMed Abstract | CrossRef Full Text | Google Scholar

Deloitte (2019). Global Mobile Consumer Survey 2019. Deloitte.

Domingues, C. M. A. S., Teixeira, A. M. d. S., and Carvalho, S. M. D. (2012). National immunization program: vaccination, compliance and pharmacovigilance. Rev. Instit. Med. Trop. de S ao Paulo 54, 22–27. doi: 10.1590/S0036-46652012000700009

PubMed Abstract | CrossRef Full Text | Google Scholar

Donzelli, G., Palomba, G., Federigi, I., Aquino, F., Cioni, L., Verani, M., et al. (2018). Misinformation on vaccination: a quantitative analysis of Youtube videos. Hum. Vaccines Immunother. 14, 1654–1659. doi: 10.1080/21645515.2018.1454572

PubMed Abstract | CrossRef Full Text | Google Scholar

Douglas, K. M., Uscinski, J. E., Sutton, R. M., Cichocka, A., Nefes, T., Ang, C. S., et al. (2019). Understanding conspiracy theories. Polit. Psychol. 40, 3–35. doi: 10.1111/pops.12568

CrossRef Full Text | Google Scholar

Dubé, E., Vivion, M., and MacDonald, N. E. (2015). Vaccine hesitancy, vaccine refusal and the anti-vaccine movement: influence, impact and implications. Expert Rev. Vaccines 14, 99–117. doi: 10.1586/14760584.2015.964212

PubMed Abstract | CrossRef Full Text | Google Scholar

Duchsherer, A., Jason, M., Platt, C. A., and Majdik, Z. P. (2020). Immunized against science: narrative community building among vaccine refusing/hesitant parents. Publ. Understand. Sci. 29, 419–435. doi: 10.1177/0963662520921537

PubMed Abstract | CrossRef Full Text | Google Scholar

Ekram, S., Debiec, K. E., Pumper, M. A., and Moreno, M. A. (2019). Content and commentary: HPV vaccine and Youtube. J. Pediatr. Adolesc. Gynecol. 32, 153–157. doi: 10.1016/j.jpag.2018.11.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Ernst, E. (2001). Rise in popularity of complementary and alternative medicine: reasons and consequences for vaccination. Vaccine 20, S90–S93. doi: 10.1016/S0264-410X(01)00290-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Evangelista, R., and Bruno, F. (2019). Whatsapp and political instability in Brazil: targeted messages and political radicalisation. Intern. Policy Rev. 8, 1–23. doi: 10.14763/2019.4.1434

CrossRef Full Text | Google Scholar

Faasse, K., Chatman, C. J., and Martin, L. R. (2016). A comparison of language use in pro- and anti-vaccination comments in response to a high profile Facebook post. Vaccine 34, 5808–5814. doi: 10.1016/j.vaccine.2016.09.029

PubMed Abstract | CrossRef Full Text | Google Scholar

G1 DF (2020). Postos de saúde do df têm Vacinas Gratuitas Contra 20 doenças; Saiba Como Colocar Caderneta em Dia. G1.

Geurkink, B. (2019). Congratulations, Youtube... Now Show Your Work. Mozilla.

Ghosh, D., and Scott, B. (2018). Digital Deceit: The Technologies Behind Precision Propaganda on the Internet. 42. New America.

Google Scholar

Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., et al. (2020). Array programming with numpy. Nature 585, 357–362. doi: 10.1038/s41586-020-2649-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Hoffman, B. L., Felter, E. M., Chu, K.-H., Shensa, A., Hermann, C., Wolynn, T., et al. (2019). It's not all about autism: the emerging landscape of anti-vaccination sentiment on Facebook. Vaccine 37, 2216–2223. doi: 10.1016/j.vaccine.2019.03.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Hunter, J. D. (2007). Matplotlib: A 2D graphics environment. Comput. Sci. Eng. 9, 90–95. doi: 10.1109/MCSE.2007.55

CrossRef Full Text | Google Scholar

Ingram, D. (2019). Youtube Pulls Ads From Anti-Vaccination Videos After Questions About Enforcement. NBC News.

Jamison, A. M., Broniatowski, D. A., Dredze, M., Wood-Doughty, Z., Khan, D., and Quinn, S. C. (2020). Vaccine-related advertising in the Facebook ad archive. Vaccine 38, 512–520. doi: 10.1016/j.vaccine.2019.10.066

PubMed Abstract | CrossRef Full Text | Google Scholar

Johnson, N. F., Velásquez, N., Restrepo, N. J., Leahy, R., Gabriel, N., Oud, S. E., et al. (2020). The online competition between pro- and anti-vaccination views. Nature 582, 230–233. doi: 10.1038/s41586-020-2281-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Jolley, D., and Douglas, K. M. (2014a). The effects of anti-vaccine conspiracy theories on vaccination intentions. PLoS ONE 9:e89177. doi: 10.1371/journal.pone.0089177

PubMed Abstract | CrossRef Full Text | Google Scholar

Jolley, D., and Douglas, K. M. (2014b). The social consequences of conspiracism: exposure to conspiracy theories decreases intentions to engage in politics and to reduce one's carbon footprint. Br. J. Psychol. 105, 35–56. doi: 10.1111/bjop.12018

PubMed Abstract | CrossRef Full Text | Google Scholar

Kata, A. (2010). A postmodern Pandora's box: anti-vaccination misinformation on the internet. Vaccine 28, 1709–1716. doi: 10.1016/j.vaccine.2009.12.022

PubMed Abstract | CrossRef Full Text | Google Scholar

Kata, A. (2012). Anti-vaccine activists, web 2.0, and the postmodern paradigm–an overview of tactics and tropes used online by the anti-vaccination movement. Vaccine 30, 3778–3789. doi: 10.1016/j.vaccine.2011.11.112

PubMed Abstract | CrossRef Full Text | Google Scholar

Keelan, J., Pavri-Garcia, V., Tomlinson, G., and Wilson, K. (2007). YouTube as a source of information on immunization: a content analysis. JAMA 298, 2482–2484. doi: 10.1001/jama.298.21.2482

PubMed Abstract | CrossRef Full Text | Google Scholar

Koltai, K. S. (2020). Human values and scientific controversies: studying vaccine information behavior on social networking sites (Ph.D. thesis). The University of Texas at Austin, Austin, TX, United States.

Lewandowsky, S., and Cook, J. (2020). The Conspiracy Theory Handbook (Fairfax, VA: George Mason University). Available online at: http://sks.to/conspiracy

Lewis, R. (2018). Alternative Influence: Broadcasting the Reactionary Right on Youtube. 61. Data & Society

Moran, M. B., Lucas, M., Everhart, K., Morgan, A., and Prickett, E. (2016). What makes anti-vaccine websites persuasive? A content analysis of techniques used by anti-vaccine websites to engender anti-vaccine sentiment. J. Commun. Healthc. 9, 151–163. doi: 10.1080/17538068.2016.1235531

CrossRef Full Text | Google Scholar

Mozilla (2019). How Healthy Is the Internet? Mozilla.

Nadler, A., Crain, M., and Donovan, J. (2018). Weaponizing the Digital Influence Machine: The Political Perils of Online Ad Tech. Data & Society.

Google Scholar

Nasir, L. (2000). Reconnoitering the antivaccination web sites: news from the front. J. Fam. Pract. 49, 731–733. Available online at: https://www.mdedge.com/familymedicine/article/60871/vaccines/reconnoitering-antivaccination-web-sites-news-front

PubMed Abstract | Google Scholar

O'Donovan, C. (2019). Youtube Just Demonetized Anti-Vax Channels. BuzzFeed News.

Oliphant, T. E. (2006). A Guide to NumPy, Vol. 1. Trelgol Publishing USA.

Google Scholar

Orr, D., Baram-Tsabari, A., and Landsman, K. (2016). Social media as a platform for health-related public debates and discussions: the polio vaccine on Facebook. Israel J. Health Pol. Res. 5:34. doi: 10.1186/s13584-016-0093-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Popken, B. (2018). As Algorithms Take Over, Youtube's Recommendations Highlight a Human Problem. NBC News.

Rallo Shimizu, N. (2020). Relaes de sentido sobre a vacinação no facebook (Master's thesis). State University of Campinas, Campinas, Brazil.

Google Scholar

Resende, G., Melo, P., Sousa, H., Messias, J., Vasconcelos, M., Almeida, J., et al. (2019). “(mis)information dissemination in Whatsapp: gathering, analyzing and countermeasures,” in The World Wide Web Conference, WWW'19 (New York, NY: Association for Computing Machinery), 818–828. doi: 10.1145/3308558.3313688

CrossRef Full Text | Google Scholar

Rieder, B. (2019). YouTube Data Tools (Version 1.11).

SciPy 1.0 Contributors, Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., et al. (2020). SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat. Methods 17, 261–272. doi: 10.1038/s41592-019-0686-2

CrossRef Full Text | Google Scholar

Shelby, A., and Ernst, K. (2013). Story and science: how providers and parents can utilize storytelling to combat anti-vaccine misinformation. Hum. Vaccines Immunother. 9, 1795–1801. doi: 10.4161/hv.24828

PubMed Abstract | CrossRef Full Text | Google Scholar

Smallman, S. (2018). Conspiracy theories and the Zika epidemic. J. Int. Glob. Stud. 9, 1–13. Available online at: https://www.lindenwood.edu/academics/beyond-the-classroom/publications/journal-of-international-global-studies/all-issues/volume-9-number-2/

Google Scholar

Smith, N., and Graham, T. (2019). Mapping the anti-vaccination movement on Facebook. Inform. Commun. Soc. 22, 1310–1327. doi: 10.1080/1369118X.2017.1418406

CrossRef Full Text | Google Scholar

Stecula, D. A., Kuru, O., and Jamieson, K. H. (2020). How trust in experts and media use affect acceptance of common anti-vaccination claims. Harvard Kennedy School (HKS) Misinform. Rev. 1, 1–11. doi: 10.37016/mr-2020-007

CrossRef Full Text | Google Scholar

Suelzer, E. M., Deal, J., Hanus, K. L., Ruggeri, B., Sieracki, R., and Witkowski, E. (2019). Assessment of citations of the retracted article by Wakefield et al. with fraudulent claims of an association between vaccination and autism. JAMA Netw. Open 2:e1915552. doi: 10.1001/jamanetworkopen.2019.15552

PubMed Abstract | CrossRef Full Text | Google Scholar

Szeto, E., Pedersen, K., and Tomlinson, A. (2020). Hidden Cameras Capture Misinformation, Fundraising Tactics Used by Anti-Vaxx Movement. CBC News.

The Tor Project Inc. (2020). Tor. The Tor Project, Inc.

Tripodi, F. (2018). Searching for Alternative Facts. Data & Society

Google Scholar

van der Linden, S. (2015). The conspiracy-effect: exposure to conspiracy theories (about global warming) decreases pro-social behavior and science acceptance. Pers. Indiv. Differ. 87, 171–173. doi: 10.1016/j.paid.2015.07.045

CrossRef Full Text | Google Scholar

van der Walt, S., Colbert, S. C., and Varoquaux, G. (2011). The numpy array: a structure for efficient numerical computation. Comput. Sci. Eng. 13, 22–30. doi: 10.1109/MCSE.2011.37

CrossRef Full Text | Google Scholar

van Rossum, G., and Drake, F. L. (2009). Python 3 Reference Manual. Scotts Valley, CA: CreateSpace.

Google Scholar

van Schalkwyk, F. (2019). “‘The amplification of uncertainty: the use of science in the social media by the anti-vaccination movement,” in Science Communication in South Africa: Reflections on Current Issues, eds P. Weingart, M. Joubert, and B. Falade (Cape Town: African Minds), 170-212. Available online at: https://zenodo.org/record/3557207#.X4ePBHVKiis

Venkatraman, A., Garg, N., and Kumar, N. (2015). Greater freedom of speech on web 2.0 correlates with dominance of views linking vaccines to autism. Vaccine 33, 1422–1425. doi: 10.1016/j.vaccine.2015.01.078

PubMed Abstract | CrossRef Full Text | Google Scholar

Ward, J. K., Peretti-Watel, P., Larson, H. J., Raude, J., and Verger, P. (2015). Vaccine-criticism on the internet: new insights based on French-speaking websites. Vaccine 33, 1063–1070. doi: 10.1016/j.vaccine.2014.12.064

PubMed Abstract | CrossRef Full Text | Google Scholar

Wolfe, R. M., Sharp, L. K., and Lipsky, M. S. (2002). Content and design attributes of antivaccination web sites. JAMA 287, 3245–3248. doi: 10.1001/jama.287.24.3245

PubMed Abstract | CrossRef Full Text | Google Scholar

World Health Organization (2019a). New Measles Surveillance Data for 2019. WHO Newsroom.

World Health Organization (2019b). Ten Threats to Global Health in 2019. WHO Newsroom.

Google Scholar

Yiannakoulias, N., Slavik, C. E., and Chase, M. (2019). Expressions of pro- and anti-vaccine sentiment on Youtube. Vaccine 37, 2057–2064. doi: 10.1016/j.vaccine.2019.03.001

PubMed Abstract | CrossRef Full Text | Google Scholar

YouTube (2020). YouTube Data API Reference. Search: List, relatedToVideoId. Google (accessed February 09, 2020).

YouTube About (2020). YouTube for Press. Google. (accessed February 09, 2020).

YouTube Help (2020). Verification Badges on Channels. Google. (accessed February 09, 2020).

Zimmerman, R. K., Wolfe, R. M., Fox, D. E., Fox, J. R., Nowalk, M. P., Troy, J. A., et al. (2005). Vaccine criticism on the World Wide Web. J. Med. Intern. Res. 7:17. doi: 10.2196/jmir.7.2.e17

PubMed Abstract | CrossRef Full Text | Google Scholar