The Effect of Frames on COVID-19 Vaccine Resistance

Abstract

In order to control the spread of infectious diseases such as COVID-19, it will be important to develop a communication strategy to counteract “vaccine resistance”, that is, the refusal to take the COVID-19 vaccine even when available. This paper reports the results of a survey experiment testing the impacts of several types of message content: the safety and efficacy of the vaccine itself, the likelihood that others will take the vaccine, and the possible role of politics in driving resistance to the vaccine. In an original survey of 1,123 American M-Turk respondents conducted in the summer of 2020, we provided six different information conditions suggesting the safety and efficacy of the vaccine, the lack of safety/efficacy of the vaccine, the suggestion that most others would take the vaccine, the suggestion that most others would not take the vaccine, the suggestion that the vaccine is being promoted by liberals to gain greater control over individual freedom, and the suggestion that its approval is being by President Trump rushed for political motivations. We compared the responses for those in the treatment groups with a control group who received no additional information. In comparison to the control group, those who received information about the safety/efficacy of the vaccine were more likely to report that they would take the vaccine, those who received information that others were reluctant to take the vaccine were more likely to report that they themselves would not take it, and those who received information about political influences on vaccine development expressed resistance to taking it. Communication of effective messages about the vaccine will be essential for public health agencies that seek to promote vaccine uptake.

Introduction

Vaccination programs have reduced the toll of infectious diseases by preventing infection or reducing the severity of symptoms, contributing to higher standards of public health by lowering morbidity and mortality rates (Andre et al., 2008). But vaccination programs are effective in providing herd immunity only when they are accepted by large segments of the population. Response to a vaccine can be understood as a continuum ranging from outright refusal to active demand for immediate uptake (Dubé et al., 2013; Larson et al., 2014). Vaccine resistance, defined as the unwillingness to get vaccinated when one is available (Lazer et al., 2021), has been identified by the World Health Organization as one of the top ten threats to global health (World Health Organization 2019; Puri et al., 2020). According to the COVID States Project, 21% of adults in the United States in February 2021 were considered “vaccine resistant”, while 31% were classified as “vaccine hesitant”, indicating that they preferred to wait until others have been vaccinated before making a personal decision on the matter (Simonson et al., 2021). In the COVID-19 pandemic of 2020–2, as in future outbreaks of vaccine-preventable illness, it will be important to combat this resistance/hesitancy and to promote vaccine uptake through effective communication strategies (Nyhan et al., 2014; French et al., 2020).

An understanding of how different types of information may influence the public’s beliefs and vaccination intentions is required in order to develop an effective communication strategy. The contribution of this paper is the examination of the causal effect of exposure to pro- and anti-vaccination message frames on individuals’ reported likelihood of getting vaccinated for COVID-19. The case study we examine here is the situation in the summer of 2020 surrounding beliefs about a vaccine for COVID-19 that had not yet been approved, but the implications are transferrable to the acceptance of vaccines developed in the future as well.

Framing Effects and Vaccination Resistance Vs. Uptake

Communication about the development and testing of any vaccine is transmitted through “frames” used in the message. For example, a media frame, or frame in communication, refers to “words, images, phrases, and presentation styles that a speaker (e.g., a politician, a media outlet) uses when relaying information about an issue or event to an audience” (Chong and Druckman 2007, 100). An emphasis framing effect occurs when exposure to a media frame causes an audience to privilege the specific consideration(s) made salient when forming an overall opinion on any issue (Druckman 2001). For example, the New York Times published an article on November 19, 2020 that announced the results from a successful COVID-19 vaccine trial with the headline, “Pfizer Says New Results Show Vaccine is Safe and 95% Effective” (Thomas 2020). Similarly, NBC News ran a story on December 9, 2020 that was headlined, “FDA: Pfizer’s COVID-19 Vaccine Safe and Effective After One Dose” (Edwards 2020). This emphasis in the news on the safety and efficacy of the vaccine and its potential to prevent mild and severe forms of COVID-19 highlights an important positive “frame”, or “subset of potentially relevant considerations”, that may influence attitudes about whether or not to get vaccinated, an important antecedent of the actual behavior (Ajzen 1991). We focus exclusively on emphasis framing effects and not equivalency framing effects that occur when positive or negative information unconsciously influences preferences (Tversky and Kahneman 1981; Druckman 2004).

The empirical study described here contributes to the understanding of the impact of message framing on vaccine resistance/uptake. We tested the impacts of several types of emphasis frames: two emphasizing the safety and efficacy (or their absence) of the vaccine, two emphasizing the likelihood that taking the vaccine would be in accord (or not) with general social norms, one suggesting that the entire discussion of vaccines is being shaped by “radical liberals” and media elites who want to exert more government control over individual behavior, and one suggesting that President Trump is pressuring the FDA to rush the approval of a COVID-19 vaccine in order to provide an “October Surprise” that might boost his chances of re-election. Given the polarizing nature of media coverage surrounding the vaccine’s development and approval process in the months leading up to August 2020 when our study was fielded, we anticipated that the effectiveness of the “political frames” might depend on whether the respondents’ political in-group was cast in a positive or a negative light.

Because of its significance to public health, there have been numerous studies of the factors that cause vaccine hesitancy and resistance (Hornsey et al., 2018; Puri et al., 2020; Thunstrom et al., 2020). Most of the studies have focused on decision-making in the context of parents vaccinating their children, the acceptance of the HPV vaccine, or decision-making with respect to uptake of the flu vaccine (Brewer et al., 2007; Dubé et al., 2013; Smith et al., 2017; Kim et al., 2019; Callaghan et al., 2020). In a review of 316 articles on framing in health communication, Guenther et al. (2020) noted that most experimental studies to date have focused on the relative persuasiveness of “gain” as opposed to “loss” frames. In other words, these studies look at the relative persuasiveness of frames that emphasize the “gains” of taking an action or compliance as opposed to the “losses” of not engaging in a particular behavior or noncompliance (e.g., applying sunscreen, scheduling a cancer screening exam, getting vaccinated). Issue frames that are applicable to a single issue or emphasis frames that emphasize different dimensions of an issue are the dominant form of political communication (Druckman 2001; Brugman and Burgers 2018). As Penta and Baban (2018) noted, there has been insufficient scholarly attention to effective messages that appear in the context of realistic news settings, and additional work is needed to better understand how to craft effective interventions that promote vaccine uptake.

Vaccine Safety and Efficacy

The influence of perceived safety on vaccine resistance has been a finding of several meta-analyses of the scientific literature. In a review of 2,791 studies published between 1990–2019, Sweileh (2020) found that although the reasons for vaccine refusal varied depending on the disease and on the cultural and national context, the overwhelming reason was fears about the safety of the vaccines. Yaqub et al. (2014) reviewed 1,187 articles published between 2009 and 2012, primarily about HPV and flu vaccines, and found that “fear of adverse side effects and vaccine safety” were the leading reasons for hesitancy or refusals, both in the general population and among healthcare professionals. Similarly, a review of 2,895 articles in English, French and Spanish from 2004–2014 (Karafillakis and Larson 2017, 4,846) found that although different concerns were expressed about vaccine safety for different types of vaccines, the “largest area of concern was vaccine safety.” In a study of childhood vaccine safety, van der Linden et al. (2015) found that agreement with a statement that “90% of medical scientists agree that vaccines are safe” was the most important predictor of public support for vaccines. Similarly, an analysis of 25 national samples from 12 different countries showed that “trust in experts” was the most consistent predictor of vaccine acceptance (Kerr et al., 2020). Finally, a more recent study found that presenting individuals with information specifically about a COVID-19 vaccine’s safety increased Americans’ plans to get vaccinated (Motta 2021).

Along with safety, the efficacy of a vaccine is also important in the decision-process (Motta 2021). In a study manipulating an H1N1 vaccination message along with perceived safety, efficacy, susceptibility to the disease and severity, Nan et al. (2012), Nan et al. (2016) found that the most important factor in the acceptance of the vaccine among older adults was perceived efficacy. Similarly, Chapman and Coups (1999) found that perceived efficacy of the flu vaccine was the most important factor in its acceptance by healthy adults, followed closely by the likelihood that it would not have side effects. News stories focused on considerations about the safety and efficacy of any COVID-19 vaccine has been persistent from the fall of 2020 to the present.

In the case of the COVID-19 vaccine, based on the large body of empirical literature emphasizing the importance of both safety and efficacy in the decision to accept a vaccine, we propose the following: Individuals presented with a message that emphasizes the safety and effectiveness of a vaccination for COVID-19 will increase their intentions to get vaccinated (Hypothesis 1a). Individuals who are presented with a message that emphasizes the lack of safety and potential ineffectiveness of any approved vaccine for COVID-19 will decrease their intentions to get vaccinated (Hypothesis 1b).

Normative Social Influence

A long scholarly tradition has demonstrated the impact of “social norms” on behavior changes. Social norms are the “tacit rules that members of a group implicitly recognize and that affect their decisions and behavior” (Brewer et al., 2017, 170). Cialdini et al. (1991) distinguished two distinct types of social norms: those that are “injunctive”, informing people about what is approved or disapproved, and those that are “descriptive” of typical or common behavior. Examples of experimental manipulations of social norms to change behavior include studies of college binge drinking (Perkins and Craig 2002), smoking (Linkenbach et al., 2003), hotel towel reuse (Goldstein et al., 2008), and energy conservation (Schultz et al., 2007; Bolsen 2013).

The underlying principle for the operation of descriptive social norms is that most people want to bring their behavior in line with what they perceive to be the behavior of others (Brewer et al., 2017). A descriptive social norms marketing campaign will therefore outline what a majority of people are doing in order to get the target audience to conform. Research has shown that “strategic messaging” that highlights a social norm can have an influence on behavioral decisions ranging from voting (Gerber et al., 2008) to using weight-loss products (Lim et al., 2020) or charitable giving (Croson et al., 2009). Indeed, the application of the scholarly findings about social norms into popular marketing has been familiar in the multitude of advertisements that suggest that “everyone else” is buying or participating in what is being sold (Melnyk et al., 2019).

Social norms may also have a negative effect on behavior because the perception that “everyone is not doing it” will decrease the intention to act (Kahan 2014, 4). In such instances, even when the intention is to increase the acceptance of a behavior, communicating that people are not adopting this behavior may have the unintended effect of decreasing engagement in the pro-social behavior being promoted (Murray and Matland 2014; Palm et al., 2020; Rimal and Real 2005; but see; Hassell and Wyler 2019).

Several studies have examined the impact of social norms on the adoption of various vaccines (Xiao and Borah 2020). Allen et al. (2009) found that social norms, that is, the perceived behavior of friends who either had already been vaccinated or were considering the vaccine, were the strongest predictors of the intent to be vaccinated against human papillomavirus (HPV). Brunson (2013) identified the role of descriptive social norms in parental decisions about their children’s vaccinations. De Bruin et al. (2019) documented the impact of perceived vaccine coverage in the social circle (defined as people with whom the respondent had regular contact) on vaccination behavior for influenza. Similarly, Parker et al. (2013) found that social influence, that is, the likelihood that people around the respondent were being vaccinated, was the most common reason for choosing to get a flu vaccine.

Because of the consensus in the literature concerning the likelihood that people will try to make their behavior conform with their perceptions of the behavior of others, we hypothesize that: Exposure to a message that emphasizes other Americans’ willingness to get vaccinated for COVID-19 will increase individuals’ intentions to get vaccinated (Hypothesis 2a). Conversely, exposure to a message that emphasizes other Americans’ unwillingness to get vaccinated for COVID-19 will decrease individuals’ intentions to get vaccinated (Hypothesis 2b).

Politicization and Vaccine Resistance/Uptake

Vaccine resistance not only has a long history, but it also reflects “historical events and individual belief systems reflective of different societal periods” (McAteer et al., 2020, 703). Not surprisingly, then, the issues around the development of a COVID-19 vaccine too have become an issue intertwined with politics in the United States. The public has become sharply divided about all aspects of the science surrounding COVID-19 from the viruses’ origin (Bolsen et al., 2020) to perceptions about the effectiveness of various government policies seeking to mitigate its impacts (Rutjens et al., 2021). A content analysis of newspaper and television coverage surrounding the issue from March to May 2020 showed that politicians were featured as often or more often than scientists (Hart et al., 2020). A recent survey in the US found that an increase in conservatism also increased the odds of vaccine resistance; moreover, those who intended to vote for President Trump in 2020 were 35% more likely to report that they would refuse a COVID-19 vaccination (Callaghan et al., 2020). Another study reported that when Republicans were exposed to an anti-vaccination argument posted on Twitter by President Trump, they became more concerned about getting vaccinated (Hornsey et al., 2020).

While conservatives have expressed higher levels of hesitancy toward a COVID-19 vaccine, President Trump stated publicly that approval of a vaccine before November would help his chances for re-election (Irfan 2020). If this rhetoric creates a perception that political interference occurred on the part of the Trump administration to pressure the FDA for rapid approval, this politicization of the vaccine may have contributed further to vaccine resistance, especially on the part of Democrats who may be motivated to view the President’s decisions about the vaccine as being driven by political goals (Taber and Lodge 2006; Bolsen and Palm 2019). The New York Times published an article voicing concerns about an October Surprise before Election Day and fear that the FDA might approve a vaccine too hastily in order to please the President (Emanuel and Offit 2020). Given this context, we hypothesize that individuals who are exposed to a “political message” stating that the COVID-19 vaccine is being rushed by President Trump for approval prior to Election Day will increase vaccine resistance (Hypothesis 3a). We anticipate that this effect is likely to be most pronounced among Democrats given their distrust toward President Trump as members of a political out-group during a divisive election.

Another political argument that surfaced in United States media was that the government should require “compulsory vaccinations” for COVID-19 “to win the war against the novel coronavirus” (Lederman et al., 2020). This rhetoric sometimes appeared alongside claims that “Operation Warp Speed” was a way to further regulate the lives of Americans and enrich drug companies. This sort of rhetoric feeds into concerns about “big government” regulating the lives of individuals and usurping personal choices about private health matters (Kavalski and Smith 2020; Dougherty 2021). We hypothesize that a message that emphasizes that the liberal media is pushing for “mandatory vaccinations” and “immunization cards” will increase vaccine resistance (Hypothesis 3b). We anticipate that this effect is likely to be most pronounced among Republicans given the information is associated as coming from the “liberal media” and being used to advance an agenda that includes greater government regulation and restrictions on personal freedom (Taber and Lodge 2006; Palm et al., 2020).

Methods and Materials

We implemented a survey-experiment in August 2020 in which we randomly assigned 1,123 respondents, recruited from Amazon’s Mechanical Turk (MTurk), to one of six experimental conditions that varied emphasis frames about a COVID-19 vaccine or to a control condition. MTurk is an online crowdsourcing platform commonly used in the social sciences to estimate causal relationships; the results are comparable to identical studies fielded on general population samples (Mullinix et al., 2015; Levay et al., 2016). We restricted the sample to United States respondents who had successfully completed at least 100 tasks and had at least a 95% approval rating on MTurk. The sample was large and diverse with respect to demographic and political characteristics: for instance, 41% of respondents identified as Republicans, 22% identified as Independents, and 37% identified as Democrats. Further, our sample is 55% female and 45% male. Other descriptive statistics for the sample are available in the Supplementary Appendix Table A1.

Experimental Treatments and Conditions

Participants in all conditions completed an IRB-approved consent form and were informed that they would be asked some questions about their opinions related to a COVID-19 vaccine. To complete the survey, respondents had to check a box to indicate they had read the following debriefing statement: “At present there is no FDA-licensed vaccine to prevent COVID-19. Vaccines have been highly effective in preventing a range of serious infectious diseases. The FDA has the scientific expertize to evaluate any potential COVID-19 vaccine candidate regardless of the technology used to produce or to administer the vaccine. This includes the different technologies such as DNA, RNA, protein and viral vectored vaccines being developed by commercial vaccine manufacturers and other entities. For factual information about the regulation of COVID-19 vaccine development, please consult this website from the Food and Drug Administration.” We also provided a link to the FDA website from which this language was drawn.

Respondents randomly assigned to the control condition (N = 157) were not exposed to any information prior to answering our key outcome measures (described below). Respondents randomly assigned to one of six other conditions were exposed to a message that was about 220 words long. The message varied the emphasis frame in the story’s headline and content of a short “article” formatted to mimic a news story about a COVID-19 vaccine. Table 1 reports the headline and full wording we incorporated into each experimental treatment. We used information from published news articles as the basis for our messages, although we edited them for length and reading level (e.g., Bump 2020; Cornwall 2020; Lederman et al., 2020; National Institutes of Health, 2020; Weixel 2020). Because we were attempting to simulate actual news articles, the “descriptive-norm” and “political” versions of our treatments also made reference to the vaccine’s safety and effectiveness to situate these frames in a realistic news-story context. In the real world, frames are often encountered in a context where a mixture of distinct or even competing frames may be present (Chong and Druckman 2007). The treatments were reflective of real-world news stories, but we acknowledge that this design makes it more difficult to isolate aspects of the stimuli that may be driving any observed impact on respondents’ vaccination intentions.

Respondents randomly assigned to the safe and effective condition (N = 172) were presented with the headline, “Scientists Are Working on a Safe and Effective COVID-19 Vaccine”, followed by information that a vaccine would be “safe, have few side effects, and most of all, will be effective in preventing the illness” and that it will have been “carefully tested and evaluated by scientists and medical professionals” (NIH 2020). Respondents assigned to the unsafe and ineffective condition (N = 159) were presented with the headline, “A COVID-19 Vaccine is Neither Effective nor Safe”, followed by an information calling into question the efficacy of any FDA-approved vaccine by noting that it will be approved by the FDA if it shows “only 50% efficacy” (FDA, 2020), suggesting that it could have serious side-effects, and that immunity could last only for a few months. Respondents in the willing condition (N = 171) were presented with the headline, “Most American Say They Will Get Vaccinated against COVID-19”, followed by information that included the results from “a recent tracking survey” that “indicated widespread willingness in the U.S.” to take the vaccine. The treatment included additional details explaining why most Americans are willing to get vaccinated (Bump 2020). Conversely, respondents in the unwilling condition (N = 157) saw the headline, “Many Americans Say They Will Not Get Vaccinated against COVID-19”, followed by information from a “recent tracking survey” that “indicated widespread reluctance in the United States to take any COVID-19 vaccination” (Cornwall 2020). It included additional details explaining why many Americans may be hesitant to take the vaccine. Two additional conditions invoked “politics” in an anti-vaccination message. In the agenda condition (N = 149), respondents were presented with the headline, “Liberal Media Pushing Agenda for ‘Mandatory Vaccinations’ and ‘Immunization Cards’’, followed by information suggesting that the rush to develop a COVID-19 vaccine is a way to enrich pharmaceutical companies and for the government to assert greater control over the lives of individuals (Lederman et al., 2020). In the Trump condition (N = 158), respondents read the headline, “President Trump Pushing for Rapid Approval of a COVID-19 Vaccine”, followed by information raising concern that the FDA might approve a vaccine due to political pressure prior to Election Day, as an “October surprise” (Cohen 2020; Weixel 2020).

The dependent variable that immediately followed exposure to one of the randomized conditions was our measure of vaccine resistance. Participants in all conditions responded to this question: “If an FDA-approved vaccine against the coronavirus becomes widely available, how likely is it that you will get vaccinated?” (1 = extremely unlikely; 7 = extremely likely)¹.

Results

To test our hypotheses, we estimate OLS regression models with robust standard errors. We regress the dependent variable (i.e., the measure for vaccine resistance/uptake) on our condition indicators, omitting the Control condition as the reference group (Table 2). We present the results with and without the inclusion of basic demographic covariates–party identification, ideology, gender, age, education, income and minority status–to improve the precision of our estimates of treatment effects and any imbalances across conditions following randomization (Angrist and Pischke 2009; Kam and Trussler 2017). In all models, cell entries contain OLS coefficients representing the difference in means between the treatment condition and the control condition. We also included a manipulation check at the end of the survey where respondents in the treatment conditions were asked if the “news article” they read earlier was opposed to or supportive of getting vaccinated for COVID-19. The treatments were accurately perceived in the directions we intended across all conditions (Supplementary Appendix Table A2).

Our first set of hypotheses tested the degree to which exposure to frames highlighting considerations related to the safety and effectiveness of a COVID-19 vaccine would shift intentions to take the vaccine in the direction of the message. As we predicted (Hypothesis 1a), respondents who read the safe and effective treatment were more likely to express an intention to get vaccinated (b = 0.36, p = 0.05, column 1, Table 2). Counter to our prediction (Hypothesis 1b), reading the not safe treatment had no statistically significant impact on respondents’ willingness to get vaccinated; however, the coefficient in the model for not safe is negative and there is some movement in the expected direction from the control condition.

Our second set of hypotheses was that frames highlighting a descriptive social norm would have an impact on respondents’ intentions regarding vaccine resistance/uptake. As we predicted (Hypothesis 2a and Hypothesis 2b), the unwilling treatment decreased reported intentions to take a COVID-19 vaccine (b = -0.40, p = 0.04, column 1, Table 2), whereas the willing treatment increased reported intentions to get vaccinated (b = 0.43, p = 0.03).

Our third set of hypotheses focused on how exposure to distinct frames that “politicize” the COVID-19 vaccine will increase vaccine resistance and decrease vaccine uptake. As we predicted (Hypothesis 3a), respondents exposed to a framed message emphasizing that President Trump is pressuring the FDA for rapid approval of a vaccine for political purposes were less likely to say that they would take the vaccine (b = −0.34, p = 0.06, column 1, Table 2). However, counter to our prediction (Hypothesis 3b), we find no evidence that the reading an article that politicizes the COVID-19 vaccine by claiming that it is being promoted by a radical liberal agenda seeking to regulate personal health decisions had any effect on respondents’ attitude regarding vaccine resistance/uptake.

We also report, in Table 2, the results from a second OLS regression model that includes the dichotomous variables for condition indicators and additional demographic and political variables that have been associated with vaccine resistance/uptake (Lazer et al., 2021; Simonson et al., 2021). First, as expected given that random assignment was successful, the substantive impact of the treatment effects is unchanged with the inclusion of these additional demographic and political measures; however, the model fit is slightly improved (and the standard errors for each condition’s estimated effect are slightly smaller) with the inclusion of these additional covariates. Second, although we must be cautious about making generalizations given the nature of our sample, we observe that both Democrats and Republicans stated that they are more likely to get vaccinated than Independents in the sample. In addition, similar to findings from surveys of nationally representative samples in the United States, minorities and females are less likely to say they will get vaccinated, while those with higher levels of education and a liberal political ideology are more likely to report a willingness to get vaccinated.

We anticipated, given the general polarization surrounding all aspects of the science of COVID-19 and the nature of our “political frames”, that party identification might moderate the impact of our treatments on respondents’ attitudes about vaccine resistance/uptake. Specifically, we anticipated the framed message highlighting President Trump pressuring for rapid approval for political purposes might have a particularly pronounced influence on Democrats, whereas we anticipated that the framed message emphasizing the role of a liberal media promoting mandatory vaccinations might be especially impactful at generating vaccine resistance among Republicans. We tested for interaction effects between the experimental condition assignment and party identification with an OLS regression (Table 3). Overall, we find almost no evidence that party identification is a moderator of any treatment effect reported in Table 2. The only statistically significant interaction term is for Republicans in the safe and effective condition: Republican respondents in this condition were less likely to say they would get the vaccine relative to other subgroups². In sum, there was almost no evidence that the “political frames” we employed had a larger effect on specific partisan subgroups in our sample, nor did most of the other messages. This may reflect the fact that with respect to COVID-19 vaccine resistance factors other than partisanship appear to play a more dominant role in resistance to the science (e.g., see Rutjens et al., 2021).

Discussion

It is crucial to know how messages the public receives about a novel vaccine ultimately shape decisions regarding whether or not to get vaccinated (Mheidly and Fares 2020). In this study, information that highlighted the safety and efficacy of an approved vaccine against COVID-19 increased individuals’ willingness to take the vaccine. As noted earlier, this study was done in the summer of 2020 before the FDA issued an emergency use authorization for any vaccine. The finding of a clear causal effect of a single exposure to a message emphasizing the vaccine’s safety and efficacy in the context of a survey suggests that as these messages are encountered in actual media stories, they may have a powerful and lasting effect on vaccine uptake.

The results also demonstrate the powerful impact that communicating descriptive social norms can exert on decisions about whether or not to get vaccinated. The experimental treatments we developed highlighted the degree to which other Americans are willing to get vaccinated for COVID-19. Responses indicating intentions to get the approved vaccine increased when respondents were informed that most Americans plan to get vaccinated in the context of a short article that included additional frames regarding the safety and efficacy of the vaccine. Learning that most Americans are hesitant about getting vaccinated, on the other hand, increased vaccine resistance. Further, we find these effects are not driven by responses among a particular partisan subgroup in our sample, but rather due to the impact of descriptive-norm-based messages across respondents. These findings suggest that messages highlighting the willingness of other Americans to get vaccinated may be an effective way to promote positive vaccination attitudes, intentions and even uptake. Future research might employ more refined treatments that parse out the role that norm-based information exerts on subgroups in the population with relatively higher levels of vaccine resistance/hesitancy, such as parents, younger and female Republican voters, ethnic minorities, and those with strong religious convictions (Lazer et al., 2021; Lin et al., 2021; Rutjens et al., 2021; Simonson et al., 2021).

We also found evidence that frames that invoked political motives of President Trump to rush the approval of the COVID-19 vaccine to help win re-election increased vaccine resistance. Although we expected that respondents who identified themselves as Democrats would be most responsive to this message, we found no difference among the partisan subgroups in the sample. With respect to the second political message, and also counter to our expectation, we found no overall or subgroup effect of the message emphasizing the vaccine being promoted to advance a liberal policy agenda and mandatory vaccinations.

We note two limitations of this study. First, the relatively small size of our partisan subgroups and nature of our convenience sample may limit the ability to reliably detect how a more representative sample of partisans might have reacted to the messages. Second, although the descriptive norm-based frames appeared in the headlines of our treatments and were also emphasized in the body of the text, the messages also mentioned the vaccine’s safety and effectiveness in order to make them seem more realistic. As such, our ability to isolate any specific word(s) or sentence(s) in the stimuli that may have been most impactful in producing the observed treatment effects is limited. Future work should also assess how people respond to simultaneous exposure to framed messages that include competing arguments, since this is more representative of the information environment in which most people live. Further, it will be important to evaluate the duration or persistence of the emphasis framing effects we uncovered.

In order to combat vaccine hesitancy, it is urgent that messaging be carefully and thoughtfully crafted, taking into account what social scientists have learned about the factors that influence message acceptance. Scientific misinformation poses a significant threat to vaccine uptake and can lead to catastrophic public health consequences (Rutjens et al., 2021). Emphasis framing is one important antidote for combatting the effects of scientific misinformation (Levy et al., 2020). We extend this research on framing and vaccine resistance/uptake by evaluating the impact of distinct theoretically motivated messages on attitudes toward getting vaccinated for COVID-19. The results demonstrate the powerful impact of perceptions regarding the safety and efficacy of any vaccine for promoting its uptake. They also suggest that conveying descriptive social norms may be a particularly fruitful avenue for targeted communication campaigns promoting vaccine uptake, for instance, among traditionally skeptical populations. It will be important for future research to identify ways to overcome the deleterious impact of messages that politicize the science surrounding any given vaccine as well, especially given the powerful role political rhetoric can play in stimulating vaccine resistance and scientific misperceptions.

References

• 1

  AjzenI. (1991). The Theory of Planned Behavior. Organizational Behav. Hum. Decis. Process.50 (2), 179–211. 10.1016/0749-5978(91)90020-t

  • CrossRef
  • Google Scholar

• 2

  AllenJ. D.MohllajeeA. P.SheltonR. C.OthusM. K. D.FontenotH. B.HannaR. (2009). Stage of Adoption of the Human Papillomavirus Vaccine Among College Women. Prev. Med.48 (5), 420–425. 10.1016/j.ypmed.2008.12.005

  • CrossRef
  • Google Scholar

• 3

  AndreF.BooyR.BockH.ClemensJ.DattaS.JohnT.et al (2008). Vaccination Greatly Reduces Disease, Disability, Death and Inequity Worldwide. Bull. World Health Org., 86, 140–146. 10.2471/blt.07

  • CrossRef
  • Google Scholar

• 4

  AngristJ. D.PischkeJ. S. (2009). Instrumental Variables in Action: Sometimes You Get what You Need. Mostly harmless Econom.113–220. 10.1515/9781400829828-009

  • CrossRef
  • Google Scholar

• 5

  BolsenT. (2013). A Light Bulb Goes on: Norms, Rhetoric, and Actions for the Public Good. Polit. Behav.35 (1), 1–20. 10.1007/s11109-011-9186-5

  • CrossRef
  • Google Scholar

• 6

  BolsenT.PalmR.KingslandJ. T. (2020). Framing the Origins of COVID-19. Sci. Commun.42 (5), 562–585. 10.1177/1075547020953603

  • CrossRef
  • Google Scholar

• 7

  BolsenT.PalmR. (2019). Motivated Reasoning and Political Decision Making. Oxford Res. encyclopedia Polit.10.1093/acrefore/9780190228637.013.923

  • CrossRef
  • Google Scholar

• 8

  BrewerN. T.ChapmanG. B.GibbonsF. X.GerrardM.McCaulK. D.WeinsteinN. D. (2007). Meta-analysis of the Relationship between Risk Perception and Health Behavior: the Example of Vaccination. Health Psychol.26 (2), 136–145. 10.1037/0278-6133.26.2.136

  • CrossRef
  • Google Scholar

• 9

  BrewerN. T.ChapmanG. B.RothmanA. J.LeaskJ.KempeA. (2017). Increasing Vaccination: Putting Psychological Science into Action. Psychol. Sci. Public Interest18 (3), 149–207. 10.1177/1529100618760521

  • CrossRef
  • Google Scholar

• 10

  BrugmanB. C.BurgersC. (2018). Political Framing across Disciplines: Evidence from 21^st Century Experiments. Res. Polit.5 (2), 1–7. 10.1177/2053168018783370

  • CrossRef
  • Google Scholar

• 11

  BrunsonE. K. (2013). The Impact of Social Networks on Parents' Vaccination Decisions. Pediatrics131 (5), e1397–e1404. 10.1542/peds.2012-2452

  • CrossRef
  • Google Scholar

• 12

  BumpP. (2020). 1 in 7 Republicans Don’t Think a Coronavirus Vaccine Is Necessary. Available at: https://www.washingtonpost.com/politics/2020/06/02/1-in-7-republicans-dont-think-coronavirus-vaccine-is-necessary/ (Accessed July 2020).

  • Google Scholar

• 13

  CallaghanT.MoghtaderiA.LueckJ. A.HotezP. J.StrychU.DorA.et al (2020). Correlates and Disparities of COVID-19 Vaccine Hesitancy. Soc Sci Med272, 113638. 10.1016/j.socscimed.2020.113638

  • CrossRef
  • Google Scholar

• 14

  ChapmanG. B.CoupsE. J. (1999). Predictors of Influenza Vaccine Acceptance Among Healthy Adults. Prev. Med.29 (4), 249–262. 10.1006/pmed.1999.0535

  • CrossRef
  • Google Scholar

• 15

  ChongD.DruckmanJ. N. (2007). Framing Public Opinion in Competitive Democracies. Am. Polit. Sci. Rev.101, 637–655. 10.1017/s0003055407070554

  • CrossRef
  • Google Scholar

• 16

  CialdiniR. B.KallgrenC. A.RenoR. R. (1991). A Focus Theory of Normative Conduct: A Theoretical Refinement and Reevaluation of the Role of Norms in Human Behavior. Adv. Exp. Soc. Psychol.24, 201–234. 10.1016/s0065-2601(08)60330-5

  • CrossRef
  • Google Scholar

• 17

  CohenJ. (2020). Here’s How the U.S. Could Release a COVID-19 Vaccine before the Election –and Why that Scares Some. Sci. Mag.28. Available at: https://www.sciencemag.org/news/2020/08/here-s-how-us-could-release-COVID-19-vaccine-election-and-why-scares-some (Accessed September 2020). 10.1126/science.abe5150August

  • CrossRef
  • Google Scholar

• 18

  CornwallW. (2020). Just 50% of Americans Plan to Get a COVID-19 Vaccine. Here’s How to Win over the Rest. Science Magazine. Available at: https://www.sciencemag.org/news/2020/06/just-50-americans-plan-get-COVID-19-vaccine-here-s-how-win-over-rest.

  • Google Scholar

• 19

  CrosonR.HandyF.ShangJ. (2009). Keeping up with the Joneses: The Relationship of Perceived Descriptive Social Norms, Social Information, and Charitable Giving. Nonprofit Manage. Leadersh.19 (4), 467–489. 10.1002/nml.232

  • CrossRef
  • Google Scholar

• 20

  De BruinB., W.ParkerA. M.GalesicM.VardavasR. (2019). Reports of Social Circles' and Own Vaccination Behavior: A National Longitudinal Survey. Health Psychol.38 (11), 975–983. 10.1037/hea0000771

  • CrossRef
  • Google Scholar

• 21

  DoughertyM. B. (2021). Vaccine Passports Are a Terrible Idea. Natl. Rev.Available at https://www.nationalreview.com/2021/03/vaccine-passports-are-a-terrible-idea/ (Accessed March 2021).

  • Google Scholar

• 22

  DruckmanJ. N. (2004). Political Preference Formation: Competition, Deliberation, and the (Ir)relevance of Framing Effects. Am. Polit. Sci. Rev.98, 671–686. 10.1017/s0003055404041413

  • CrossRef
  • Google Scholar

• 23

  DruckmanJ. N. (2001). The Implications of Framing Effects for Citizen Competence. Polit. Behav.23 (3), 225–256. 10.1023/a:1015006907312

  • CrossRef
  • Google Scholar

• 24

  DubéE.LabergeC.GuayM.BramadatP.RoyR.BettingerJ. A. (2013). Vaccine Hesitancy. Hum. Vaccin. Immunother.9 (8), 1763–1773. 10.4161/hv.24657

  • CrossRef
  • Google Scholar

• 25

  EdwardsE. (2020). FDA: Pfizer’s COVID-19 Vaccine Safe and Effective after One Dose. Available at: https://www.nbcnews.com/health/health-news/fda-pfizer-s-COVID-19-vaccine- safe-effective-after-one-n1250337 December 8.

  • Google Scholar

• 26

  EmanuelE. J.OffitP. A. (2020). Could Trump Turn a Vaccine into a Campaign Stunt. New York Times. Available at: https://www.nytimes.com/2020/06/08/opinion/trump-coronavirus-vaccine.html (Accessed July 2020).

  • Google Scholar

• 27

  FDA (2020). COVID-19-Related Guidance Documents for Industry, FDA Staff, and Other Stakeholders. available at https://www.fda.gov/media/139638/download (Accessed July 2020).

  • Google Scholar

• 28

  FrenchJ.DeshpandeS.EvansW.ObregonR. (2020). Key Guidelines in Developing a Pre-emptive COVID-19 Vaccination Uptake Promotion Strategy. Ijerph17 (16), 5893. 10.3390/ijerph17165893

  • CrossRef
  • Google Scholar

• 29

  GerberA. S.GreenD. P.LarimerC. W. (2008). Social Pressure and Voter Turnout: Evidence from a Large-Scale Field Experiment. Am. Polit. Sci. Rev.102, 33–48. 10.1017/s000305540808009x

  • CrossRef
  • Google Scholar

• 30

  GoldsteinN. J.CialdiniR. B.GriskeviciusV. (2008). A Room with a Viewpoint: Using Social Norms to Motivate Environmental Conservation in Hotels. J. Consum Res.35 (3), 472–482. 10.1086/586910

  • CrossRef
  • Google Scholar

• 31

  GuentherL.GaertnerM.ZeitzJ. (2020). Framing as a Concept for Health Communication: A Systematic Review. Health Commun.29, 1–9. 10.1080/10410236.2020.1723048

  • CrossRef
  • Google Scholar

• 32

  HartP. S.ChinnS.SorokaS. (2020). Politicization and Polarization in COVID-19 News Coverage. Sci. Commun.42, 679–697. 10.1177/1075547020950735

  • CrossRef
  • Google Scholar

• 33

  HassellH. J. G.WylerE. E. (2019). Negative Descriptive Social Norms and Political Action: People Aren't Acting, So You Should. Polit. Behav.41 (1), 231–256. 10.1007/s11109-018-9450-z

  • CrossRef
  • Google Scholar

• 34

  HornseyM. J.FinlaysonM.ChatwoodG.BegenyC. T. (2020). Donald Trump and Vaccination: The Effect of Political Identity, Conspiracist Ideation and Presidential Tweets on Vaccine Hesitancy. J. Exp. Soc. Psychol.88, 103947. 10.1016/j.jesp.2019.103947

  • CrossRef
  • Google Scholar

• 35

  HornseyM. J.HarrisE. A.FieldingK. S. (2018). The Psychological Roots of Anti-vaccination Attitudes: A 24-nation Investigation. Health Psychol.37 (4), 307–315. 10.1037/hea0000586

  • CrossRef
  • Google Scholar

• 36

  IrfanU. (2020). Why It’s Unlikely We’ll Have a COVID-19 Vaccine before Election Day. Available at: https://www.vox.com/2020/8/20/21359026/coronavirus-vaccine-COVID-19-when-october-surprise-election-trump (Accessed November 2020).

  • Google Scholar

• 37

  KahanD. M. (2014). Vaccine Risk Perceptions and Ad Hoc Risk Communication: An Empirical Assessment. CCP Risk Perception Studies Report No. 17, Yale Law & Economics Research Paper # 491. Available at SSRN: https://ssrn.com/abstract=2386034 or http://dx.doi.org/10.2139/ssrn.2386034 (Accessed July 2020).

  • Google Scholar

• 38

  KamC. D.TrusslerM. J. (2017). At the Nexus of Observational and Experimental Research: Theory, Specification, and Analysis of Experiments with Heterogeneous Treatment Effects. Polit. Behav.39 (4), 789–815. 10.1007/s11109-016-9379-z

  • CrossRef
  • Google Scholar

• 39

  KarafillakisE.LarsonH. J. (2017). The Benefit of the Doubt or Doubts over Benefits? A Systematic Literature Review of Perceived Risks of Vaccines in European Populations. Vaccine35 (37), 4840–4850. 10.1016/j.vaccine.2017.07.061

  • CrossRef
  • Google Scholar

• 40

  KavalskiE.SmithN. R. (2020). Immunity Passports: a “New” Old Idea with Baggage. Glob. Pol. J.Available at: https://www.globalpolicyjournal.com/blog/27/04/2020/immunity-passports-new-old-idea-baggage (Accessed July 2020).

  • Google Scholar

• 41

  KerrJ. R.SchneiderC. R.RecchiaG.DryhurstS.SahlinU.DufouilC.et al (2020). Predictors of COVID-19 Vaccine Acceptance across Time and Countries. medRxiv preprint.

  • Google Scholar

• 42

  KimS.PjesivacI.JinY. (2019). Effects of Message Framing on Influenza Vaccination: Understanding the Role of Risk Disclosure, Perceived Vaccine Efficacy, and Felt Ambivalence. Health Commun.34 (1), 21–30. 10.1080/10410236.2017.1384353

  • CrossRef
  • Google Scholar

• 43

  LarsonH. J.JarrettC.EckersbergerE.SmithD. M. D.PatersonP. (2014). Understanding Vaccine Hesitancy Around Vaccines and Vaccination from a Global Perspective: A Systematic Review of Published Literature, 2007-2012. Vaccine32 (19), 2150–2159. 10.1016/j.vaccine.2014.01.081

  • CrossRef
  • Google Scholar

• 44

  LazerD.OgnyanovaK.GreenJ.MatthewA.DruckmanJ.GitomerA.et al (2021). The COVID States Project: A 50-State COVID-19 Survey Report #47: Update on COVID-19 Vaccine Attitudes Among Healthcare Workers. Available at: www.covidstates.org (Accessed March 2021). 10.31219/osf.io/arwh3

  • CrossRef
  • Google Scholar

• 45

  LedermanM.MehimanM. J.YoungnerS. (2020). Defeat COVID-19 by Requiring Vaccination for All. It’s Not un-American, It’s Patriotic. USA Today. Available at: https://www.usatoday.com/story/opinion/2020/08/06/stop-coronavirus-compulsory-universal-vaccination-column/3289948001/(Accessed August 6, 2021).

  • Google Scholar

• 46

  LevayK. E.FreeseJ.DruckmanJ. N. (2016). The Demographic and Political Composition of Mechanical Turk Samples. Sage Open6 (1), 2158244016636433. 10.1177/2158244016636433

  • CrossRef
  • Google Scholar

• 47

  LevyJeremy.BayesRobin.BolsenToby.DruckmanJames. N. (2020). Science and the Politics of Misinformation. Handbook of media, misinformation, and populism23, 77–94. 10.1146/annurev-polisci-050718-032814

  • CrossRef
  • Google Scholar

• 48

  LimJ. S.ChockT. M.GolanG. J. (2020). Consumer Perceptions of Online Advertising of Weight Loss Products: the Role of Social Norms and Perceived Deception. J. Marketing Commun.26 (2), 145–165. 10.1080/13527266.2018.1469543

  • CrossRef
  • Google Scholar

• 49

  LinC.TuP.BeitschL. M. (2021). Confidence and Receptivity for COVID-18 Vaccines: a Rapid Systematic Review. Vaccines9 (1), 16. 10.3390/vaccines9010016

  • CrossRef
  • Google Scholar

• 50

  LinkenbachJ. W.PerkinsH.DeJongW. (2003). “Parents' Perceptions of Parenting Norms: Using the Social Norms Approach to Reinforce Effective Parenting,” in The Social Norms Approach to Preventing School and College Age Substance Abuse: A Handbook for Educators, Counselors, and Clinicians. Editor PerkinsH. W. (Jossey-Bass/Wiley), 247–258.

  • Google Scholar

• 51

  McAteerJ.YildirimI.ChahroudiA. (2020). The VACCINES Act: Deciphering Vaccine Hesitancy in the Time of COVID-19. Clin. Infect. Dis.71, 703–705. 10.1093/cid/ciaa433

  • CrossRef
  • Google Scholar

• 52

  MelnykV.Van HerpenE.JakS.Van TrijpH. C. M. (2019). The Mechanisms of Social Norms' Influence on Consumer Decision Making. Z. für Psychol.227 (1), 4–17. 10.1027/2151-2604/a000352

  • CrossRef
  • Google Scholar

• 53

  MheidlyN.FaresJ. (2020). Leveraging Media and Health Communication Strategies to Overcome the COVID-19 Infodemic. J. Public Health Pol.41, 410–420. 10.1057/s41271-020-00247-w

  • CrossRef
  • Google Scholar

• 54

  MottaM. (2021). Can a COVID-19 Vaccine Live up to Americans' Expectations? A Conjoint Analysis of How Vaccine Characteristics Influence Vaccination Intentions. Soc. Sci. Med.272, 113642. 10.1016/j.socscimed.2020.113642

  • CrossRef
  • Google Scholar

• 55

  MullinixK. J.LeeperT. J.DruckmanJ. N.FreeseJ. (2015). The Generalizability of Survey Experiments. J. Exp. Polit. Sci.2 (2), 109–138. 10.1017/xps.2015.19

  • CrossRef
  • Google Scholar

• 56

  MurrayG. R.MatlandR. E. (2014). Mobilization Effects Using Mail. Polit. Res. Q.67 (2), 304–319. 10.1177/1065912913499234

  • CrossRef
  • Google Scholar

• 57

  NanX.MaddenK.RichardsA.HoltC.WangM. Q.TracyK. (2016). Message Framing, Perceived Susceptibility, and Intentions to Vaccinate Children against HPV Among African American Parents. Health Commun.31 (7), 798–805. 10.1080/10410236.2015.1005280

  • CrossRef
  • Google Scholar

• 58

  NanX.XieB.MaddenK. (2012). Acceptability of the H1N1 Vaccine Among Older Adults: the Interplay of Message Framing and Perceived Vaccine Safety and Efficacy. Health Commun.27 (6), 559–568. 10.1080/10410236.2011.617243

  • CrossRef
  • Google Scholar

• 59

  National Institutes of Health (2020). Experimental Coronavirus Vaccine Is Safe and Produces Immune Response. Available at: https://www.nih.gov/news-events/nih-research-matters/experimental-coronavirus-vaccine-safe-produces-immune-response (Accessed July 2021).

  • Google Scholar

• 60

  NyhanB.ReiflerJ.RicheyS.FreedG. L. (2014). Effective Messages in Vaccine Promotion: a Randomized Trial. Pediatrics133 (4), e835–e842. 10.1542/peds.2013-2365

  • CrossRef
  • Google Scholar

• 61

  PalmR.BolsenT.KingslandJ. T. (2020a). "Don't Tell Me what to Do": Resistance to Climate Change Messages Suggesting Behavior Changes. Weather, Clim. Soc.12 (4), 827–835. 10.1175/wcas-d-19-0141.1

  • CrossRef
  • Google Scholar

• 62

  PalmR.BolsenT.KingslandJ. (2020b). The Effect of Frames on COVID-19 Vaccine Hesitancy. medRxiv preprint. (Accessed January 2021)

  • Google Scholar

• 63

  ParkerA. M.VardavasR.MarcumC. S.GidengilC. A. (2013). Conscious Consideration of Herd Immunity in Influenza Vaccination Decisions. Am. J. Prev. Med.45 (1), 118–121. 10.1016/j.amepre.2013.02.016

  • CrossRef
  • Google Scholar

• 64

  PentaM. ABabanA. (2018). Message Framing in Vaccine Communication: A Systematic Review of Published Literature. Health Commun.33 (3), 299–314. 10.1080/10410236.2016.1266574.

  • Google Scholar

• 65

  PerkinsH.CraigD. W. (2002). A Multifaceted Social Norms Approach to Reduce High-Risk Drinking: Lessons from Hobart and Williams Smith Colleges. Newton: Higher Education Center for Alcohol and Other Drug Prevention.

  • Google Scholar

• 66

  PuriN.CoomesE. A.HaghbayanH.GunaratneK. (2020). Social Media and Vaccine Hesitancy: New Updates for the Era of COVID-19 and Globalized Infectious Diseases. Hum. Vaccin. Immunother., 16(11), 2586–2593. 10.1080/21645515.2020.1780846

  • CrossRef
  • Google Scholar

• 67

  RimalR. N.RealK. (2005). How Behaviors Are Influenced by Perceived Norms. Commun. Res.32 (3), 389–414. 10.1177/0093650205275385

  • CrossRef
  • Google Scholar

• 68

  RutjensB. T.van der LindenS.van der LeeR. (2021). Science Skepticism in Times of COVID-19. Group Process. Intergroup Relations24 (2), 276–283. 10.1177/1368430220981415

  • CrossRef
  • Google Scholar

• 69

  SchultzP. W.NolanJ. M.CialdiniR. B.GoldsteinN. J.GriskeviciusV. (2007). The Constructive, Destructive, and Reconstructive Power of Social Norms. Psychol. Sci.18 (5), 429–434. 10.1111/j.1467-9280.2007.01917.x

  • CrossRef
  • Google Scholar

• 70

  SimonsonM.BaumM. A.LazerD.OgnyanovaK.GitomerA.PerlisR. H.et al (2021). The COVID States Project Report #45: COVID-19 Vaccine Hesitancy and Resistance Among Parents. Available at: https://osf.io/e95bc/ (Accessed March 2021).

  • Google Scholar

• 71

  SmithL. E.AmlôtR.WeinmanJ.YiendJ.RubinG. J. (2017). A Systematic Review of Factors Affecting Vaccine Uptake in Young Children. Vaccine35 (45), 6059–6069. 10.1016/j.vaccine.2017.09.046

  • CrossRef
  • Google Scholar

• 72

  SweilehW. M. (2020). Bibliometric Analysis of Global Scientific Literature on Vaccine Hesitancy in Peer-Reviewed Journals (1990–2019). BMC Public Health20 (1), 1–15. 10.1186/s12889-020-09368-z

  • CrossRef
  • Google Scholar

• 73

  TaberC. S.LodgeM. (2006). Motivated Skepticism in the Evaluation of Political Beliefs. Am. J. Polit. Sci.50 (3), 755–769. 10.1111/j.1540-5907.2006.00214.x

  • CrossRef
  • Google Scholar

• 74

  ThomasK. (2020). Pfizer Says New Results Show Vaccine Is Safe and 95% Effective. The New York Times. Available at: https://www.nytimes.com/2020/11/18/health/pfizer-covid-vaccine.html (Accessed November 2019).

  • Google Scholar

• 75

  ThunstromL.MadisonA.FinnoffD.NewboldS. (2020). Hesitancy towards a COVID-19 Vaccine and Prospects for Herd Immunity. Available at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3593098 (Accessed August 2021).

  • Google Scholar

• 76

  TverskyA.KahnemanD. (1981). The Framing of Decisions and the Psychology of Choice. Science211 (4481), 453–458. 10.1126/science.7455683

  • CrossRef
  • Google Scholar

• 77

  van der LindenS. L.ClarkeC. E.MaibachE. W. (2015). Highlighting Consensus Among Medical Scientists Increases Public Support for Vaccines: Evidence from a Randomized Experiment. BMC Public Health15 (1), 1–5. 10.1186/s12889-015-2541-4

  • CrossRef
  • Google Scholar

• 78

  WeixelN. (2020). Some Worry ‘Operation Warp Speed’ Plays into Anti-vaccination Movement’s Hands. Available at: https://thehill.com/policy/healthcare/500213-some-worry-operation-warp-speed-plays-into-anti-vaccination-movements-hands (Accessed July 2020).

  • Google Scholar

• 79

  World Health Organization (2019). Vaccination: European Commission and World Health Organization Join Forces to Promote the Benefits of Vaccines. Available at: https://www.who.int/news/item/12-09-2019-vaccination-european-commission-and-world-health-organization-join-forces-to-promote-the-benefits-of-vaccines (Accessed July 2020).

  • Google Scholar

• 80

  XiaoX.BorahP. (2020). Do Norms Matter? Examining Norm-Based Messages in HPV Vaccination Promotion. Health Commun.26, 1–9. 10.1080/10410236.2020.1770506

  • CrossRef
  • Google Scholar

• 81

  YaqubO.Castle-ClarkeS.SevdalisN.ChatawayJ. (2014). Attitudes to Vaccination: a Critical Review. Soc. Sci. Med.112, 1–11. 10.1016/j.socscimed.2014.04.018

  • CrossRef
  • Google Scholar