Natural Stings: Selling Distrust About Vaccines on Brazilian YouTube

In this study, we investigate misinformation and disinformation (M&D) about vaccines using a case study approach to understand how M&D about vaccines circulate on YouTube in Portuguese, and who are the channels creating and disseminating this kind of content. The World Health Organization considered vaccine hesitation as one of the greatest threats to global health in 2019. Researchers associated this hesitation to a strengthening of the anti-vaccination movements, suggesting that social media is currently the main spreader of this position. YouTube increasingly becomes a matter of concern, since its recommendation system is identified as a promoter of misinformation and extreme content. Despite YouTube's statements, M&D about vaccines continue to be disseminated in videos in Portuguese, reaching a large audience. We found 52 videos containing M&D about vaccines. The main M&D were the claim of dangerous ingredients in vaccines, the defense of self-direction—freedom of choice, independent research—, the promotion of alternative health services, the myth that vaccines cause diseases, conspiracy theories, and the allegation of vaccine's severe collateral effects. We identified 39 brands advertising on 13 videos of our M&D sample. Although the YouTube Partner Program is an important source of income, the channels use different economic strategies, such as the selling of courses, and therapies and the use of fundraising platforms. We also found that alternative health channels spread distrust about traditional institutions to promote themselves as trusted sources for the audience and thereby profit with alternative health services.


INTRODUCTION
The World Health Organization considered vaccine hesitation as one of the greatest threats to global health in 2019 (World Health Organization, 2019b). With the decrease in immunization coverage, 170 countries registered cases of measles, increasing 300% when compared to the first three months of 2018 (World Health Organization, 2019a). In this scenario, Brazil was the sixth country by reported cases of measles in 2019 (Castelvecchi et al., 2019), despite the National Immunization Program, one of the strongest Brazilian public health policy programs (Domingues et al., 2012), offering 20 different types of vaccines free of charge to the population (G1 DF, 2020). Researchers associated the decrease in immunization to a strengthening of the anti-vaccination movements (Dubé et al., 2015;Benecke and DeYoung, 2019), suggesting that social media is currently the main spreader of this position (Stecula et al., 2020).
YouTube increasingly becomes one of the most popular social media services, reaching over two billion users each month (YouTube About, 2020). YouTube's recommendation system is responsible for 70% of total viewing time, and drives the promotion of certain content within the site (Popken, 2018). YouTube's reach is a reason for concern, since its recommendation system is identified as a promoter of misinformation and extreme content (Mozilla, 2019;Avaaz, 2020). There is still a gap in the research literature on vaccine M&D in other languages than English and when investigating this topic, great part of the researchers draw attention to other platforms, such as Facebook and Twitter. As suggested by Allgaier (2018), investigating YouTube empirically is still challenging because of the numerous approaches and problematic aspects associated with it, reinforcing the importance of research aimed at this online environment.
After a series of complaints from academics and journalists (Geurkink, 2019), YouTube stated the implementation of measures to reduce recommendations of harmful misinformation and to remove advertisements from antivaccination videos (Ingram, 2019). Such claims can not be confirmed without access to its data. Also, these measures are even more complex when considering content in languages other than English and countries without proper regulation. For instance, YouTube has great influence in the spread of anti-vaccination content in Portuguese: almost 90% of the videos analyzed in Avaaz and SBIm (2019) presented some misinformation about vaccines, reaching 7.4 mi of visualizations in the last three years. These are concerning results, since 57% of the interviewed subjects that did not vaccinate themselves or their children declared misinformation about vaccines as a reason (Avaaz and SBIm, 2019).
In this paper, we investigated M&D about vaccines using a case study approach. We used a sample obtained from the terms "vacina+autismo" (vaccine+autism, in English), to understand how M&D about vaccines circulate on YouTube in Portuguese, and who are the channels creating and disseminating this kind of content. These terms were used due to the prevalence of the false link between vaccines and autism (Suelzer et al., 2019), a disinformation spread first in 1998 with the publication of a fraudulent paper (Deer, 2011) and still used as an argument by the anti-vaccination movements (Venkatraman et al., 2015;van Schalkwyk, 2019).
Despite YouTube's statements on their fight against misinformation (O'Donovan, 2019), we verified that channels already identified as creators of harmful M&D (Avaaz and SBIm, 2019) continue to offer questionable videos to the public, with many of them still being part of the YouTube Partner Program. In our sample, we found a community of content creators that collaborate to promote alternative health services meanwhile spread M&D about vaccines. These channels have financial gains selling courses, books, and alternative treatments, requesting donations through fundraising platforms and deposits in bank accounts, and even from large companies, through advertisements on YouTube.

Acquiring and Pre-processing the Data
We used YouTube Data Tools (Rieder, 2019) to obtain the data used in this study. The tool was set up using the option "Video Network, " which creates a network of relations between videos via YouTube's "related videos" feature: YouTube defines "related videos" from the relatedToVideoId parameter, which contains a list of videos related to a given video . We selected the following parameters on YouTube Data Tools: • Search query: "vacina+autismo" (in English, "vaccine+autism") • Iterations: 1 • Rank by: relevance • Crawl depth: 2 The data was obtained in triplicate on February 10, 2020, between 19:00 and 21:00 UTC. This way, we obtained 1,714, 2,135, and 2,286 videos in each extraction. Then, we combined the three datasets and used Gephi (Bastian et al., 2009) to obtain specific videos: containing the word vacina-vaccine-in their title; connected to the network, with a network degree higher than one-i.e., connected to the network by more than one related video; with more than 10,000 views. We used the following queries on Gephi: Giant Component → Degree Range: ≥ 2 → Range (viewcount): → 10000 These processes reduced the data to 191 videos. From these videos, we excluded the irrelevant ones-information on animal vaccination, technical material addressed to health workers, advertisements and videos on the Vaccine Revolt, a historical moment in Brazil. These videos were watched and analyzed completely, and were considered as irrelevant due to their content; for example, videos on the Vaccine Revolt were created by official institutions or science communication channels that approach the event from a historical perspective. Then we reached the initial sample of our study, consisting of 158 videos.
We performed the classification on the initial sample between February 11 and 18, 2020, using a protocol of vaccine misinformation and disinformation (M&D) developed for this research. All videos were watched completely; when vaccine M&D was identified, the content was manually classified by an author (DFTM). If any of the videos created uncertainty, a second author (LG) would analyze the material and both would discuss to reach consensus. Several videos had more than one category of M&D about vaccines. We classified M&D categories in each video, and our classification is presented in the Supplementary Material (de Siqueira, 2020) in order of appearance. We also identified and documented the brands advertising on each video manually throughout screenshots, available-along with the Supplementary Material-on a GitHub repository (de Siqueira, 2020). We watched the videos using the Tor browser (The Tor Project, Inc, 2020), to minimize YouTube's algorithm system personalization.
We also created a secondary sample visiting the channels containing M&D videos from the initial sample between March 3 and 11, 2020. The videos selected contained the word "vacina, " or related to vacina-e.g., VACINAS: Você acredita Nelas? (VACCINES: Do you believe in them?)-in the title. These videos were classified between March 12 and 20, 2020, using the same protocol from the initial sample. Therefore, our sample has 52 videos containing M&D about vaccines in total-23 from the initial sample and 29 from the second sample.

Classifying the Videos
To specify what videos on our sample contain misinformation or disinformation, we labeled them according to six main categories created for this study. Each category consists of the main M&D spread about vaccines: 1. Safety: vaccines cause diseases (A); vaccinated people transmit the disease (B); vaccines cause autism (C); vaccines cause severe collateral effects (D); vaccines contain dangerous ingredients (E); it is safer to contract the disease than vaccinate (F); vaccines can impair the immune system (G); overload on vaccines and its consequences (H); alternative vaccination schedules are safer (I) 2. Effectivity: vaccines do not work (J); vaccines are not responsible for the decrease in diseases (K) 3. Alternative health: promotion of alternatives to vaccination, mainly naturopathic and wellness services (L) 4. Morality: association between HPV vaccine and promiscuity, and/or religious issues (M) 5. Conspiracy theories: narratives about powerful institutions or actors with nefarious intentions and secret plans (N) 6. Other: self-direction-freedom of choice, independent research (O); claim that people that know the "truth" do not vaccinate (P); emotional appeal (Q) These categories are based on myths about vaccination most heard by Brazilian physicians, most spread M&D about vaccines in Brazil (Avaaz and SBIm, 2019), and in research analyzing the content of sites or videos about vaccination on the last two decades (Nasir, 2000;Wolfe et al., 2002;Zimmerman et al., 2005;Kata, 2010;Bean, 2011;Ward et al., 2015;Ekram et al., 2019;Yiannakoulias et al., 2019). Self-direction, as presented in the category Other, is based on Koltai's research on the antivaccination movement in the United States (Koltai, 2020).

License and Reusability
The algorithms and functions implemented in this study were written in Python (van Rossum and Drake, 2009), using the packages Numpy (Oliphant, 2006;van der Walt et al., 2011;Harris et al., 2020), Scipy (SciPy 1.0 Contributors et al., 2020), and Matplotlib (Hunter, 2007). All code and data published with this paper is available under the BSD 3-clause license, and all figures generated in this paper are available under the CC-BY 4.0 license.

RESULTS
Our initial sample contained 158 videos. We classified 23 videos from this sample with at least one misinformation or disinformation ( To characterize how the amount of M&D in a video is related to its duration and the engagement it received, we defined an engagement metric, ε, based on the number of views, comments, likes, and dislikes in the videos containing M&D in both samples: We could apply the engagement metric in 43 of the 52 videos, since the other ones do not make their numbers of likes, dislikes, or comments available. The largest part of our sample that contains M&D-38 videos out of 43-has <40 min and 20% of engagement (Figure 1). Considering both samples, 32 videos referred to vaccines in general. Some discuss specific vaccines: 14 of them mentioned flu vaccines; 12 cited the yellow fever vaccine; seven discussed the HPV vaccine; two referred to the measles vaccine and yet meningitis B, varicella, rubella, smallpox, diphtheria, polio, whooping cough, and tuberculosis vaccines were each mentioned in one video. An interesting find is the reference to a cancer vaccine in four videos of the sample. According to the content producers, vitamin D and sun exposure would be the "real" cancer vaccine, a claim that appears in videos that promote alternative services in regard to vaccination.
We identified 39 brands advertising on at least one M&D video in our sample through YouTube: 5econds, 7 Springs  Another relevant aspect is the number of verified channels in the sample. Verified channels are official channels of a creator, artist, company, or public figure. Channel verification helps distinguishing official channels from other ones with similar names (YouTube Help, 2020). Eight channels, from 20 in our sample, are verified ( Table 1): two are associated with broadcast TV programs, while the other six are self-made amateur videos.
The main M&D examples spread by the eight verified channels are the promotion of alternatives to vaccination (L)-present in seven channels-, the defense of self-direction (O)-cited in five channels-, and the allegation of dangerous ingredients in vaccines (E)-mentioned in four channels. The channels are also subscribed to different YouTube categories, indicating that M&D about vaccines can appear in several content spaces; for example, the eight verified channels have categories ranging from "Education" to "People & Blogs" (Table 1).
There is a collaboration between channels that promote alternative health services. From 20 channels spreading M&D about vaccines, 11 mentioned Lair Ribeiro (Dr. Lair Ribeiro Oficial), a cardiologist and nutrologist that promotes alternative therapies, diets, and pseudoscience-homeopathy, detox, law of attraction, quantum medicine-in his videos and talks. The collaboration occurs through the reproduction of videos from "associate" channels or via endorsement of content creators and their courses (Figure 2). Besides that, the channels promote other professionals that support alternative therapies or other content creators that endorse M&D about vaccines.
One strategy used by six channels is to ask testimonials from the audience, aiming to demonstrate the effectiveness of alternative therapies or courses they promote. Besides sending testimonials, the public has access to other services using social media platforms such as WhatsApp or Telegram. Some channels send daily "health tips, " offer discounts to products and claim that the use of these services are necessary to ensure that the public will receive new contents. Five of 20 channels make these communication services available, and one channel maintains 10 WhatsApp groups.
We also analyzed the upload date from the videos containing M&D in our sample to verify if YouTube could be in the process of removing videos from the sample that could have been recently uploaded. Twenty three videos were uploaded in 2018; seven were uploaded in 2017, and six were uploaded in 2019. The

DISCUSSION
The belief that vaccines contain dangerous ingredients (E), the "toxin gambit, " persists between the anti-vaccination movements (Kata, 2012). The videos mention specific ingredients such as aluminum, mercury, formaldehyde, thimerosal, nagalase, aborted fetal tissues, poison, toxins, and chemicals. This belief is associated with two other M&D from the Safety categoryvaccines cause diseases (A) and vaccines cause severe collateral effects (D). The narrative of unsafe vaccines is strengthened by the popularity of conspiracy theories (N), sowing doubt about public health organizations, physicians, scientists, universities, and mainstream media.
Conspiracy theories are attempts to explain events and circumstances from secret plots by powerful and malevolent actors (Douglas et al., 2019). The spread of conspiracy theories is a matter of concern, since exposition to conspiracy theories may have negative consequences, such as decreasing science acceptance (van der Linden, 2015), reduction in intention to engage in politics (Jolley and Douglas, 2014b) and, in the specific case of anti-vaccine conspiracy theories, can affect vaccination intentions (Jolley and Douglas, 2014a) and the pursuit of alternative vaccination schedules (Callaghan et al., 2019). The issue of mistrust-of vaccines, vaccination providers, or policy makers-is considered a driver for vaccine hesitation (Brown et al., 2018). Thus, the spread of distrust acts as a strategy to promoters of alternative therapies: in doing so, they affirm themselves as trusted sources for the audience and create potential consumers to the courses, books, and therapies they sell. The association between vaccine opposition and alternative health services has been made several times before (Ernst, 2001;Busse et al., 2008;Kata, 2010); for instance, Caulfield et al. (2017) analyzed 330 naturopath websites to understand the role of complementary and alternative medicine providers on the vaccine hesitation, and found that 40 of them presented antivaccination rhetoric, while 26 promoted vaccine alternatives.
The channels offered alternatives for vaccines and for treatments of health problems in a narrative of return to nature, as seen before by Kata (2012). Expressions like "God's pharmacy" and "disease's industry" are repeated to denominate the opposition between the right option-alternative products and therapies-and the "unnatural, " wrong one-vaccines, drugs, and mainstream medicine. "Disease's industry" is a reference to one of the most reiterated arguments of antivaccination conspiracy theories, where governments, scientists and physicians would be involved in a conspiracy with pharmaceutical companies to cause suffering and to profit (Jolley and Douglas, 2014a). Therefore, the advocates for alternative health services present themselves simultaneously as brave antagonists and victims persecuted for their resistance to a "corrupt system" (Lewandowsky and Cook, 2020).
A common strategy used by the channels is the production of free materials such as recorded talks, courses and e-books. The offering of free content helps to reinforce the idea of a selfless mission where the creators have no financial interests and is also a tactic to attract and retain an audience for the channels. Two of the content creators, Ribeiro and Bussadean orthomolecular physician and nutrologist that promotes the chemical compound known as Miracle Mineral Solution and vitamins as treatments for autism-affirm that they do not practice medicine, but continue their studies and produce content because they are motivated by the mission of helping people to find good recommendations of natural health and cures for their sufferings. This mission is reinforced by their claims of persecution, an argument present in vaccination opposed spaces (Avaaz and SBIm, 2019;Smith and Graham, 2019;Koltai, 2020). Ribeiro, for example, criticizes the unpreparedness of traditional physicians and accuses the pharmaceutical industry. Kata (2012) describes this trope as the "Galileo gambit, " an argument that invokes the attacks from the scientific orthodoxy of the past against scientists such as Galileo Galilei-the claim is that the currently persecuted ideas will eventually be accepted as truth.
The alleged persecution cited by the content creators in our sample is also associated with the recent changes in social media policies to contain the spread of M&D. The content creators adopt workarounds to the moderation practices, as seen on anti-vaccine communities on Facebook (Koltai, 2020), for instance. One of the workarounds of the alternative health channels is using other social media platforms, such as Telegram or WhatsApp, to ensure the content circulation and the perception of community in and out of YouTube. WhatsApp is one of the most popular information resources for Brazilians (Deloitte, 2019), and it has favored the spread of health and political misinformation in the last years (Smallman, 2018;Evangelista and Bruno, 2019;Resende et al., 2019). In addition to enabling constant interactions with community members, these WhatsApp and Telegram groups encourage the sharing of audience testimonials, which are used in new videos aiming to demonstrate the effectiveness of alternative therapies and courses promoted by the channels. Testimonials or anecdotes are also an important tactic used by anti-vaccination communities (Davies et al., 2002;Zimmerman et al., 2005;Kata, 2010;Shelby and Ernst, 2013;Ward et al., 2015;Duchsherer et al., 2020;Rallo Shimizu, 2020), possibly impacting the perception of risk about vaccines (Betsch et al., 2011). Apart from the sample's six channels asking for testimonials from the audience, another one used a dramatic testimonial of a young girl to alarm the viewers against the HPV vaccine.
According to the vaccine opposed conspiracy narrative, people need to exercise their self-direction-as in freedom of choice and independent research-to protect themselves and their families from malevolent and powerful actors. Freedom, choice and individuality are values associated with antivaccination beliefs (Moran et al., 2016). The emphasis on freedom of choice is a strategy used by vaccine opponents to rebrand the anti-vaccination movement and to avoid being censored by platforms (Koltai, 2020). The need for independent research about vaccines-as in research for yourself; the ability to do research for oneself and to ask questions is cited as a condition to freedom (Koltai, 2020)-is a recurrent argument among vaccine opposed actors to advocate for more informed choices (Hoffman et al., 2019;Duchsherer et al., 2020). However, the correct way to conduct independent research is already set by the alternative health community narratives-since they are part of a conspiracy plan, mainstream media, traditional physicians, and public health organizations cannot be trusted. Meanwhile, the constant collaboration between channels that promote alternative health services affirm content creators as reliable sources for their viewers, strengthening a sense of community and belonging (Koltai, 2020;Rallo Shimizu, 2020) that favors the creation of a loyal audience. Duchsherer et al. (2020) analyzed parent testimonials from the anti-vaccination documentary Vaxxed and found an association between community building and the division of people and institutions in two groups: trustworthy-community members such as parents who do not vaccinate, like-minded online groups, and alternative medical professionals-and adversaries-governmental agencies and health providers that endorse vaccination protocols.
The collaboration into the vaccine M&D network resembles the behavior of the Alternative Influence Network (AIN), as described by Lewis (2018). The AIN is a content creators network on YouTube that is driven by a "set of shared ideas about progressive politics and social justice" and that uses, among other things, referrals, and guest appearances of different creators in videos as an strategy to build an audience (Lewis, 2018). Beyond that, the AIN spreads distrust about the mainstream media to present itself as an "alternative" media system, claims an alleged persecution because of their beliefs, encourages the audience to do "their own research, " and uses different strategies to monetize content. According to Tripodi (2018), besides enabling the reach of new and larger audiences, the network strategy helps to reaffirm the same narratives and positions between channels.
An important aspect of the spreading dynamics of M&D about vaccines is the adoption of different strategies to profit. Although researchers already identified advertisements running on climate misinformation videos (Allgaier, 2019;Avaaz, 2020) and on videos disseminating M&D about vaccines (Avaaz and SBIm, 2019), and despite YouTube's statements about measures to fight harmful misinformation on the platform (O'Donovan, 2019), we found that monetization of these kind of content still persists. We identified 39 brands advertising on 13 videos of our M&D sample. Besides global brands such as Mobil, Kia, Fiat, Philips, Spotify, Eucerin (Beiersdorf), and Buscopan (Boehringer Ingelheim), we found advertisements from the governments of India and Japan. We also highlight the presence of advertisements from companies of alternative health and wellness products-Boiron, Lume Deodorant, Adler Pharma, and Health and Wellness Tools. It is possible that the brands could be unaware that they are helping to fund channels that spread M&D about vaccines, although alternative health advertisers are potentially reaching the audience they want to reach.
Although the YouTube Partner Program is an important source of income, it is not the only funding option for content creators (Alexander, 2019). They maintain profiles on fundraising platforms or sell merchandise as well, and selling products and courses is a regular activity, particularly between alternative health channels (Avaaz and SBIm, 2019). For example, the host of the channel nutrição Alimentos & Cia encourages the audience to buy alternative health courses in a large number of their videos. To validate these courses, he stresses the adoption of alternative health practices by the Brazilian Unified Health System. Despite YouTube's statements on their fight against misinformation, M&D about vaccines continue to be disseminated in videos in Portuguese, reaching a large audience. Even when channels are identified as creators of harmful misinformation, their videos continue to be available to the public and can be disseminated in other platforms, such as WhatsApp, as we found out in this research. For example, five channels from our samplenutrição Alimentos & Cia, O Melhor Para Você, U MIÓ QUE TÁ TENO, Dr. Lair Ribeiro Oficial, and Ana Paula Palagar -were already identified by Avaaz and SBIm (2019) and all of them are part of the collaboration's network described in this study (Figure 2). We found that M&D about vaccines are associated with alternative health channels that spread distrust about traditional institutions-public health organizations, physicians, scientists, universities, and mainstream mediato promote and profit with alternative health services. YouTube must adopt transparent approaches to counter M&D and ensure that content creators follow the user policies. The platform also needs to guarantee that M&D will not be financially stimulated through the YouTube Partner Program.
Special attention should be given for languages other than English, since YouTube's current policies to fight harmful content do not seem to be applied to these languages. For that, YouTube needs to have a qualified team of human content moderators for different countries and languages. Our research suggests that their automated filters are not capable of identifying certain types of harmful content in Portuguese. Kata (2012), Koltai (2020), and Szeto et al. (2020), for instance, identified content creators that spread misinformation using strategies such as the replacement of "problematic terms, " to prevent the platforms to recognize content that does not follow the platform policy, an extra challenge for automatic systems. In addition to the urgence to ensure that videos with M&D do not be incentivized and rewarded, advertisers need to track the content that their publicity revenue is funding and need to ask for effective mechanisms to exclude their ads from harmful content. Other specific responses are frequently suggested (e.g., Caplan et al., 2018;Ghosh and Scott, 2018;Lewis, 2018;Nadler et al., 2018;Mozilla, 2019;Avaaz, 2020), and should be considered by YouTube, since the collaboration with researchers could lead to better policies and mechanisms that ensure that effective policies are implemented and respected.

DATA AVAILABILITY STATEMENT
The data generated for this study can be found on GitHub: https://git.io/JJfVK.

ETHICS STATEMENT
Written informed consent was not obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.