Exposure to COVID-19 Vaccine Hesitancy Is as Impactful as Vaccine Misinformation in Inducing a Decline in Vaccination Intentions in New Zealand: Results from Pre-Post Between-Groups Randomized Block Experiment

While recent studies have investigated how health messages on vaccine characteristics shift public intentions to get a COVID-19 vaccine, a few studies investigate the impact of real-world, widely shared vaccine misinformation on COVID-19 vaccine acceptance. Moreover, there is currently no research that investigates how exposure to hesitancy, as compared to misinformation, is associated with COVID-19 vaccination intentions. Based on data from a nationally representative survey experiment conducted in March 2021 (N = 1,083), exposure to outright COVID-19 vaccine misinformation as well as exposure to vaccine hesitancy induces a decline in COVID-19 vaccination intentions to protect self and to get the vaccine to protect others in New Zealand, compared to factual information from government authorities. Moreover, there is no significant difference in exposure to misinformation or hesitancy in the self-reported change in COVID-19 vaccination intentions. However, respondents are more likely to believe in vaccine hesitancy information and share such information with family and followers compared to misinformation. Implications for research in health communication campaigns on COVID-19 are presented.


INTRODUCTION
As of May 2021, more than 170 million people have been infected and 3.54 million have died due to the coronavirus disease  according to the World Health Organization (https://covid19. who.int). In an unprecedented scientific endeavor, a number of COVID-19 vaccines have been developed and are being currently administered in several countries only 1 year after the COVID-19 pandemic was declared in January 2020 (Zimmer et al., 2021).
Yet the challenge of public vaccination rates prevails with a significant proportion of the population hesitant or skeptical about the safety and the benefits of vaccination (de Figueiredo et al., 2020;Fadda et al., 2020;Palamenghi et al., 2020;Verger and Dubé, 2020;Motta et al., 2021). In a survey conducted in 19 countries, differences in COVID-19 vaccine acceptance ranged from 90% in Asian countries such as China to less than 55% in countries such as Russia (Lazarus et al., 2021). Health experts say that a high proportion of the population-with estimates ranging from 60% to 85%-need to be vaccinated to achieve herd or population immunity (Sanche, 2020).
A primary concern of health experts and governments is low public acceptance of a COVID-19 vaccine, even as new variants continue to add pressure on the healthcare workers. There is increasingly scholarly interest to identify factors that drive  vaccination intentions so that health communication campaigns can address public concerns and improve vaccination rates (Kerr and Wilson, 2018;Freeman et al., 2021;Motta et al., 2021;Schwarzinger et al., 2021). Public concerns about COVID-19 vaccines include perceived side effects of the vaccine, long-term impact of vaccination, and perceived rush to approve a vaccine (Hotez et al., 2021).
These concerns are aided by widely circulating false or misleading information on social media platforms about the origins of the virus, distrust with government and experts, and risks and long-term effects of COVID-19 vaccine, among others (Vicario et al., 2016;Burki, 2019;Tokojima Machado et al., 2020). Up to two million flyers containing some of these allegations about the COVID-19 vaccines have been dropped in letterboxes across New Zealand by the "Voices of Freedom" group in May 2021 (Doyle, 2021).
New Zealand has been declared as one of the best countries that has dealt with the COVID-19 pandemic through tough border controls, contact tracing, and effective health communication (Roy, 2020;World Health Organization, 2020;Hong et al., 2021). Yet, about 40% of the population are either hesitant or skeptical towards COVID-19 vaccines (Authors). According to the New Zealand government's Ministry of Health surveys, about 10% of the population say they will "definitely not" take a vaccine, with a further 20% unlikely to take one. Further 11% are unsure to take a vaccine (COVID-19: Vaccine research insights, 2021). The New Zealand government plans to start vaccinating the general public only in July 2021 and the current low rates of COVID-19 vaccination intent poses a challenge to the government's largest vaccination program.
This study contributes to health communication research on COVID-19 in three unique ways. First, a majority of previous research has largely focused on how respondents' demographic characteristics, values, beliefs, and knowledge are associated with COVID-19 vaccination intentions (Romer and Jamieson, 2020;Roozenbeek, 2020). Similarly, researchers have tested how labmanufactured health messages shape public COVID-19 vaccination intentions, mostly through post-exposure items only Kerr et al., 2021;Motta et al., 2021;Schwarzinger et al., 2021). While important, they do not reflect real-world "infodemic"-the enormous amount of information including false or misleading information on COVID-19. Second, there are very few studies focusing on the impact of exposure to real-world misinformation on vaccination intentions using more robust pre-post experimental studies (Loomba et al., 2021). This study aims to bridge this gap.
Second, researchers have ignored if exposure to outright vaccine misinformation compared to exposure to vaccine hesitancy information differentially impacts public intentions for a COVID-19 vaccine. It is possible that outright vaccine misinformation, such as COVID-19 interfering with DNA, may be construed as unreal and rejected by a majority of the public. Yet, vaccine hesitancy information, such as the perceived rush to develop a COVID-19 vaccine compared to the long delay in developing other vaccines, may be construed as true and likely impact health behaviors. Indeed, Loomba and others (2021) found that exposure to information that highlights scientific imagery or links had the most impact on lowering vaccination intent among the public in the United States and United Kingdom.
Based on nationally representative data, this study tests if the impact of misinformation will be different from the impact of hesitancy information on COVID-19 vaccine intentions when compared to exposure to factual information by government health authorities. Moreover, as previous studies indicate that altruistic messaging could affect public acceptance of a vaccine [Loomba et al., 2021; also see (Rabb et al., 2021)], this study further differentiates between intentions to vaccinate to protect oneself and intentions to vaccinate to protect vulnerable groups, family, and friends.
For the purpose of this research, COVID-19 misinformation refers to false or misleading information that contradicts the best expert evidence available at the time (Vraga and Bode, 2020). In this study, widely circulated social media post about COVID-19 vaccines interfering with DNA is operationalized as misinformation. COVID-19 vaccine hesitancy is defined as reluctance or refusal to vaccinate despite its accessibility (MacDonald, 2015) and is operationalized in this study as information showcasing an individual's story about hesitancy to a COVID-19 vaccine who is otherwise vaccine compliant. The key difference is that hesitancy may appear to be more authentic and accurate than compared to outright misinformation, thereby impacting vaccination intentions.

MATERIALS AND METHODS
A nationally representative sample survey of the New Zealand adults (N 1,083) was conducted between February 15 and March 6, 2021, to coincide with the vaccine rollout (https://www. health.govt.nz/our-work/diseases-and-conditions/covid-19novel-coronavirus/covid-19-data-and-statistics/covid-19vaccine-data). The web-based survey was fielded by Qualtrics, an international survey agency. Qualtrics maintains an active online panel that is representative of the target population (Qualtrics Panel, n.d.). The panelists receive an incentive such as a flat fee or discount gift card based on the length of the survey, their specific profile, and target acquisition difficulty.
Respondent quotas were set to match the official Census estimates on age, gender, education, and ethnicity. The sample demographics closely matched with the Census estimates on gender, age, and ethnicity but had a higher proportion of educated and fewer Pasifika respondents. The average time to complete the survey was 22 min. Ethics approval was filed at the (Massey) University's human research review board and the study was determined to be exempt from a full review. Participants provided informed consent after reading the brief aims of the survey, namely, to understand public opinion about current issues facing the country and the world.

Study Design
The study design was a pre-post within-and between-groups design with a total of three conditions. Prior to the assignment, all respondents answered COVID-19 vaccination intentions. The respondents then answered other questions unrelated to the primary goal of the survey as a "distraction," specifically questions related to sustainability attitudes and behaviors. Based on equal proportions of gender, ethnicity, education, and income, participants were randomly assigned to one of the three conditions: misinformation, hesitancy, and factual condition. After the exposure, participants were asked about their intentions to vaccinate against COVID-19 and the following post-only measures of self-reported change, believability, fact-check, and share information. The sample characteristics between the experimental and control conditions are given in Table 1 and show no significant differences in demographics between the three groups.
After the survey, participants were debriefed with the following statement: "The aim of this study was to monitor your perceptions towards a COVID-19 vaccine and to assess whether the images we showed you changed your perceptions towards vaccinating. Some of the images we showed you are examples of online information that may contain either misleading or incorrect information about a COVID-19 vaccine. For up-to-date information surrounding the COVID-19 pandemic, please consult the government's official information page: https://covid19.govt.nz/

Selection of Images
We surveyed popular misinformation on COVID-19 vaccines in New Zealand using the AFP Fact Check website for New Zealand and Pacific Islands that documents and corrects misinformation on a variety of issues (https://factcheck.afp.com/afp-newzealand-and-pacific-islands). The following were chosen based on the criteria of prominence and popularity: 1) a widely shared vaccine misinformation social media post, included by a New Zealand political party, that the vaccine will interfere with genetic material (New Zealand party spreads misinformation about Covid-19 vaccines, 2020) and 2) a newspaper article in the national press detailing a mother's concern to vaccinate son against COVID-19 who is otherwise vaccine compliant (Rindelaub, 2021). For control, we used a social media post from "Unite Against COVID," a government Facebook page, announcing vaccine approval and rollout (see Supplementary Information).
The first misinformation post was selected because it was not only shared widely in New Zealand but also popular worldwide. The Center for Countering Digital Hate (2021) identified the source, Dr. Robert F. Kennedy Jr, as the second most important "Disinformation Dozen" spreading vaccine misinformation online. The second newspaper article is sourced from the most circulated newspaper in New Zealand, The New Zealand Herald, read by 54 percent of the population (New Zealand Herald, 2021). As such, the news would have been read widely by the New Zealand public. The third image was the first government announcement about the safety and regulatory approval of the vaccine. While these selected experimental stimuli are not exhaustive of the universe of misinformation, hesitancy, and factual information from the government, they provide a model to test its impact on public intentions.
Respondents were primed to treat the exposure as information shared on social media: "Please read the information presented in the image and answer the following questions. While answering these questions, imagine that the image were shared by your friends, followers, or people you follow on any social media platform that you use."

Measures
The primary outcome was the post-treatment measure of COVID-19 vaccination intention. The pre-treatment and post-treatment measures of COVID-19 vaccination intentions were the same and were adopted from previous studies (e.g., Loomba et al., 2021). Specifically, the respondents were asked, "When a coronavirus (COVID-19) vaccine becomes available-Would you accept the vaccine for yourself?" The responses were measured on a fourpoint scale from "Yes, definitely" (1), "Unsure, but leaning towards Yes" (2), "Unsure, but leaning towards No" (3), and "No, definitely not" (4). The mean pre-test score to vaccinate to protect oneself was 1.73 (SD 0.99) and the mean post-test score was 1.56 (SD 1.78). A higher score indicates higher vaccine hesitancy.
Similarly, respondents were asked, "When a coronavirus (COVID-19) vaccine becomes available-Would you accept the vaccine if it meant protecting friends, family, or at-risk groups?" The responses options were the same as above. The mean pre-test score was 1.56 (SD 0.86) and mean post-test score was 1.66 (SD 0.92).
The pre-test scores for intention to vaccinate to protect self and to protect others were significantly correlated (r 0.87, p < 0.001). Similarly, post-test scores were also significantly correlated (r 0.83, p < 0.001). All the four measures of pre-post scores to vaccinate for self and others were strongly correlated with each other (r's ranged from 0.79 to 0.87, p < 0.001).

Self-Reported Change in Vaccination Intention
After exposure to the three conditions, respondents were asked "Overall, the information provided in this image makes me...." The responses were measured on a five-point scale from, "Much less inclined to be vaccinated" (1), "A little less inclined to be vaccinated" (2), "No less or more inclined to get vaccinated" (3), "A little more inclined to get vaccinated" (4), and "Much more inclined to get vaccinated" (5).

Believability of Information
Respondents were asked, "Overall, how likely are you to believe the information presented in this image." The responses were measured on a five-point scale from "Extremely likely" (1) to "Extremely unlikely" (5). The responses were reverse coded so that a higher value would indicate respondents were more likely to believe in the information presented.

Fact-Check Information
Respondents were asked, "Overall, how likely are you to factcheck the information in this image via other sources?" The responses were measured on a five-point scale from, "Extremely likely" (1) to "Extremely unlikely" (5). The responses were reverse coded so that a higher value would indicate greater intention to fact-check the information.

Share Information
Respondents were asked, "Overall, how likely are you to share this image with your friends or followers?" The responses were measured on a five-point scale from "Extremely likely" (1) to "Extremely unlikely" (5). The responses were reverse coded so that a higher value would indicate greater intention to share information.

Statistical Analyses
To check for similar composition between the conditions on demographics and intentions on COVID-19 vaccine preexposure, a one-way analysis of variance (ANOVA) was used. To test differences across the conditions on intentions to take a vaccine to protect self was statistically significant, an analysis of covariance (ANCOVA) was conducted with the post-test vaccination intention as the dependent variable, experimental condition as the independent variable, and the pre-test vaccination intention as the covariate. A similar strategy was followed to test if exposure to experimental conditions differentially impacted intentions to take a vaccine to protect others. Additional robustness checks were conducted using ANOVA on post-test only measures of self-reported change in vaccination intention, believability in information, intentions to fact-check, and willingness to share information.
Less than 1% of the data contained missing values, with the largest missing values for willingness to share information (n 16): these respondents were equally distributed across the misinformation (n 5), hesitancy (n 7), and control (n 4) conditions. According to Jakobsen et al. (2017, p.3) "Complete case analysis may be used as the primary analysis if the proportions of missing data are below approximately 5%...." Therefore, missing values were excluded from the analysis.
These analyses were conducted using IBM SPSS version 27.

RESULTS
There was no significant difference in gender, age, education, and ethnicity between the three conditions (see Table 1). Similarly, there was no significant difference between the three conditions on pre-test measure of intention to vaccinate against COVID-19 to protect self [F (2, 1,079) 1.10, p 0.33] and intention to vaccinate against COVID-19 to protect others [F (2, 1,079) 1.61, p 0.20]. These findings show that the composition of the three groups was similar on demographics and COVID-19 vaccination intentions before exposure. As a manipulation check, respondents were asked, "Which of the following statements best represents information in the image?" The response options were, "Information is supportive of getting vaccinated for COVID-19" and "Information is opposed to getting vaccinated against COVID-19." There was a significant difference between the two experimental conditions and control on correctly identifying if the information in the image was supportive or opposed to getting vaccinated against COVID-19 [χ 2 (2) 436.008, p <. 001]. A large majority of respondents in both the misinformation (75%) and hesitancy condition (64%) said that the information is opposed to getting vaccinated, whereas almost all respondents in the factual information control (97%) said information was supportive of getting vaccinated against COVID-19. Descriptive within-subject differences in COVID-19 vaccination intention for self are reported in Figure 1 and COVID-19 vaccination intention to protect others are shown in Figure 2. They show that both the misinformation and hesitancy exposure conditions increased vaccine hesitancy compared to the control group.
The ANCOVA revealed a significant main effect of the treatments on COVID-19 vaccination intention to protect self [F (2, 1,074) 7.50, p < 0.001, η p 2 0.02]. Post hoc multiple comparisons using the Bonferroni adjusted marginal means indicated that vaccine hesitancy was higher in the misinformation condition (x 0.11, SE 0.03, p < 0.01, d 0.14) compared to the control. Vaccine hesitancy was also higher in the hesitancy condition compared to the control (x 0.11, SE 0.03, p < 0.01, d 0.21). There was no significant difference between the misinformation and hesitancy conditions (x 0.004, p 1.00, d 0.06).
In addition, there was no change in the above reported results after accounting for demographic variables. Among demographic variables, the more educated were less hesitant (B −0.05, SE 0.01, t −3.361, p < 0.001). Compared to European New Zealanders, the Pasifika were more hesitant to take a COVID-19 vaccine to protect themselves (B −0.22, SE 0.07, t −3.14, p < 0.01). There was no significant difference for the Māori (B 0.04, SE 0.04, p 0.31) and Asians (B −0.01, SE 0.03, p 0.72), compared to European New Zealanders. There was no significant association between gender (B 0.01, SE 0.03, p 0.63), age (B −0.01, SE 0.01, p 0.38) and income (B −0.02, SE 0.01, p 0.05) with intention to vaccinate to protect self.
Similarly, ANCOVA revealed a significant main effect of the treatments on COVID-19 vaccination intentions to protect others [F (2, 1,072) 5.74, p < 0.01, η p 2 0.01]. Post hoc multiple comparisons indicated that vaccine hesitancy was higher in the misinformation condition (x 0.12, SE 0.04, p < 0.01, d 0.15) compared to the control. Vaccine hesitancy was also higher in the hesitancy condition compared to the control (x 0.11, SE 0.04, p < 0.05, d 0.20). There was no significant difference between the misinformation and hesitancy conditions (x 0.01, p 1.00, d 0.04).
There was no change in the above findings after including demographic variables in the model. Among demographic variables, only age was significantly associated with intentions to vaccinate to protect others, with older respondents less likely to vaccinate to protect others compared to younger respondents (B −0.02, SE 0.01, t −2.43, p < 0.05). Gender (B −0.02, SE 0.03, p 0.57), education (B −0.03, SE 0.02, p 0.08), income (B −0.01, SE 0.01, p 0.25), and ethnicity was not significantly associated with intention to vaccinate to protect others.
As additional robustness checks, we examined association between exposure to treatment conditions and several post-test only measures. There was a significant main effect of experimental conditions on self-reported change in willingness to get a COVID-19 vaccine, F (2, 1,064) 32.70, p < 0.001. Post hoc comparisons using Games-Howell test indicated that perceived change in willingness to get a COVID-19 vaccine was lower in misinformation (M 3.10, SD 1.15, p < 0.001, d 0.49) and hesitancy (M 3.06, SD 0.97, p < 0.001, d 0.57) conditions compared to the control condition (M 3.64, SD 1.09), respectively. There was no significant difference between the misinformation and hesitancy conditions (p 0.89).
There was a significant main effect of condition on believability of information presented in the image, F (2, 1,076) 171.79 p < 0.001. Post hoc comparisons indicated that believability was the lowest in misinformation (M 2.53, SD 1.36, p < 0.001, d 1.33) and hesitancy (M 3.31, SD 1.05, p < 0.001, d 0.79) conditions compared to the control condition (M 4.11, SD 0.98). There was also significant difference between the misinformation and hesitancy conditions (p < 0.001, d 0.64) such that the respondents in the hesitancy conditions were more likely to believe in the information presented in the image. There was no significant difference between the conditions

DISCUSSION
A unique contribution of this study is that not only exposure to outright vaccine misinformation (COVID-19 vaccine interfering with DNA) but also exposure to vaccine hesitancy can reduce COVID-19 vaccination intentions to protect self and protect others, compared to control. This finding is robust and is based on pre-post change in COVID-19 vaccination intention as well as post-test only selfreported change in vaccination intention measure. In addition, there was no difference between exposure to hesitancy information and exposure to vaccine misinformation in the selfreported change in vaccination intentions.
Previous research has largely tested if exposure to labgenerated persuasive health messages results in a change in vaccination intentions, including vaccine characteristics and benefits for self and others Kerr et al., 2021;Motta et al., 2021;Schwarzinger et al., 2021). Moreover, a majority of these studies focus on post-test measures only, which may not control for differences in intentions prior to exposure. In contrast, a few studies examine how real-world vaccine information exposure affects vaccination intentions (Loomba et al., 2021). Exposure to vaccine hesitancy information is as potent as exposure to outright misinformation in reducing public intentions for a COVID-19 vaccine.
Respondents who were exposed to hesitant information were more likely to believe the information and share it among family and friends compared to respondents exposed to misinformation, however. While beyond the scope of this research, one of the possible pathways through which hesitancy affects vaccine intentions is by being more believable compared to outright misinformation, thereby increasing the shareability of information among family and friends. Previous research indicates that social norms or perceived popularity of behaviors (descriptive norms) and expectations from significant others to follow such behaviors (injunctive norms) play an important role in public health behaviors (Rimal and Lapinski, 2015;Rimal and Storey, 2020). Dissemination of information among social networks is likely to affect how individuals perceive what significant others are doing, thereby affecting their own behavior. For example, Motta and others (2021) found that "ordinary people" communicating the personal risks and collective benefits of COVID-19 vaccination were comparatively more effective than medical experts communicating such messages.

Limitations
One of the limitations of the study is the lack of equivalence between the message conditions, even though all the respondents were primed to respond to them as seen in their social media feed. Unlike lab experiments, real-world misinformation about vaccines is likely to be different in content, structure, and visuals. Nonetheless, future research should test for equivalency between treatment conditions for exposure to hesitancy and exposure to misinformation on vaccination intentions. Moreover, none of the previous studies evaluated the longitudinal impact of exposure to misinformation and hesitancy, including this article. There is a need to evaluate how exposure to misinformation and hesitancy affects in the long term. Previous research indicates that even brief exposure to misinformation can lead to long-term false memories (Zhu, 2012). Therefore, there is a need to continuously assess if intentions shift with the vaccine rollout as news stories about evidence about the efficacy of the vaccine as well anecdotal stories about vaccine side effects are likely to influence vaccination intentions. Another limitation is the Hawthorne effect, which we tried to minimize by asking questions unrelated to the study between the preand post-vaccine intention measures so that participants are less likely to remember their initial choice. Previous meta-analyses indicate the risk of such bias on health-related behaviors may be small and overestimated (Rodrigues et al., 2015). Future research can ask respondents after a few days or weeks of being exposed to the experimental stimulus. Finally, additional variables of attitudes towards vaccines, trust in government, and social norms should be evaluated in future research.

CONCLUSION
Implications of this study are important not just for New Zealand, which has scheduled vaccination for the general public starting in July 2021, but also for other countries where after the initial rush, vaccination rates have been stagnant (Caldwell, 2021;Pelley and Warren, 2021). For example, Israel (Law, 2021) was one of the first countries to quickly vaccinate its citizens but the proportion of the population receiving at least one dose of vaccine has been at 60% from March 2021 (https://ourworldindata.org/covidvaccinations). Reaching herd or population immunity now lies in the arms of the vaccine-hesitant. Misinformation correction campaigns should not only focus on outright misinformation but also on vaccine hesitancy, such as side effects, the need for vaccination across age groups, and clarifying the extremely rare risks associated with vaccines. Public health officials should not hesitate to transparently inform people of the limitations of vaccinations either, providing them with an authoritative resource rather than driving them to alternative sources that may misinform the public. A recent study shows that transparent communication does not reduce intentions to be vaccinated against COVID-19 (Kerr et al., 2021).
Messages addressing individuals' concerns, through messengers they trust, are likely to help overcome vaccine hesitancy and improve COVID-19 vaccination rates (Motta et al., 2021).

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

ETHICS STATEMENT
This project has been evaluated by peer review and judged to be low risk. Consequently, it has not been reviewed by one of the University's Human Ethics Committees. The researchers named in this document are responsible for the ethical conduct of this research. If you have any concerns about the conduct of this research that you want to raise with someone other than the researchers, please contact Professor Craig Johnson, Director-Ethics, telephone: 06 3569099 ext 85271, email: humanethics@ massey.ac.nz. The patients/participants provided their written informed consent to participate in this study.

AUTHOR CONTRIBUTIONS
JT conceived the study, was responsible for the fielding of the survey, and wrote the article. AS helped with the data analysis.

FUNDING
Massey University provided funding for data collection.