The Acceptance of COVID-19 Vaccination Under Different Methods of Investigation: Based on Online and On-Site Surveys in China

Abstract

As Coronavirus Disease-2019 (COVID-19) vaccines became available in December 2020, increasingly more surveys were organized to examine the acceptance of vaccination, while most of them were conducted online. This study aimed to explore the difference between online and traditional on-site surveys in terms of COVID-19 vaccine acceptance. From November to December 2020, an online survey (n = 2013) and an on-site survey (n = 4,316) were conducted simultaneously in China. Multivariate logistic regression was used to identify influencing factors of acceptance, and propensity score matching (PSM) was adopted to balance the outcomes. As a result, 90.0% of the online respondents accepted COVID-19 vaccination, while it was only 82.1% in the on-site survey. After applying PSM, the acceptance rate of the on-site survey was declined to 78.6%. The age structure, residence location, education, and health status were observed as important factors in addressing vaccination acceptance, which needed to be specifically considered when designing online surveys.

Introduction

Since firstly identified in December 2019, Coronavirus Disease-2019 (COVID-19) has spread globally with a severe situation (1–3). The pandemic has resulted in 177.1 million confirmed cases and 3.8 million deaths worldwide as of June 18, 2021 (2), which might have profound impacts on healthcare systems and public health management mechanisms (4, 5). With debates intensifying about lockdowns around the world, vaccines were regarded as the most effective weapon to effectively control the public health crisis. The research and development of vaccines against COVID-19 have been accelerating at an unprecedented speed. By November 2020, multiple candidate vaccines had been tested in the final stage (6), such as those developed by Pfizer-BioNTech, Oxford-AstraZeneca, Sinopharm, and Sinovac (7, 8). It was announced by the Chinese government on December 31, 2020 that COVID-19 vaccination would be available free of charge for Chinese citizens. This achievement has provided confidence to the global fight against COVID-19, which has also provided strong support to ease the pandemic in China and return to normal economic development.

The success of the vaccination program is dependent on the willingness of the general public to get vaccinated. A small number of studies have surveyed adults to estimate public acceptance of COVID-19 vaccination online during the early period of the pandemic. For example, the acceptance rates could reach around 90% in China, 85% in Brazil, 80% in South Africa and South Korea, which in Russia (55%) and France (60%) were lower (9–13). The pandemic in most countries was still severe with some difficulties in conducting on-site surveys, as the spread rate and infection rate among the countries and regions worldwide have continuously been on the rise. Based on the weekly report of the European Center for Disease Prevention and Control (ECDC) as on October 14, 2021, there have been 238,460,430 reported cases and a total of 4,855,764 deaths recorded in 219 countries, territories, and international conveyance (14, 15). Under this situation, most countries had implemented strict measures to control the pandemic, such as lockdowns, restriction in the movements and gathering, social distancing, and quarantining (16). Currently, with the “second-wave” of COVID-19 cases, some governments have considered and implemented further lockdowns to control the pandemic, such as the USA, Italy, Spain, France, and Australia (17, 18). Due to the recurrence of the COVID-19 pandemic and the restrictions in social distancing, an online survey becomes the most ideal method to survey and obtain vaccination acceptance. Although hit by COVID-19 at the first, China has effectively brought the pandemic under control (19–23). Due to the alleviation of the pandemic by joint efforts of the Chinese government and citizens, large-scale on-site surveys became attainable in China in the last quarter of 2020.

As explained in a previous review, online surveys can be subject to considerable bias. Bias can be resulted from the non-representative nature of the online participants with self-selection under the inevitable volunteer effect (24). Since two important components of survey methodology, sample selection and question validation, are frequently overlooked in online surveys, results generated online may be neither replicable nor robust (25). Due to the social distancing requirement in most countries, conducting online surveys is the most appropriate method during the pandemic, enabled with flexibility, automation, timeliness, and lower cost (26). However, the lack of face-to-face interviewers could become a disadvantage with an accumulation of non-representative or biased responses, which is exactly what traditional on-site surveys could address.

Therefore, this study investigated the differences in individual acceptance generated from an online and an on-site survey toward the COVID-19 vaccination, which were conducted simultaneously in China. This study is the very first survey conducted in China and in the world at the very beginning of vaccination approval and before roll-out in China and globally. The aim of this study was to compare the online and on-site field survey results toward the public acceptance of COVID-19 vaccination, which is also groundbreaking with the first large-scale field survey in China and in the world to investigate the public acceptance toward COVID-19 vaccination. It should be noted that the acceptance rate addressed in this study might be different from the up-to-date acceptance rate after nearly a year of routine vaccination. By examining the results under the two survey methods, we targeted to alert global researchers of the potential biases in online surveys and emphasize the principles of survey research, which should be applied in online surveys. More practically, the appropriate statistical methodology would be suggested in this study to reduce bias and enhance rigor, which could be adjusted to more accurate and close-to-reality results when conducting an online survey during the pandemic.

Materials and Methods

Study Design, Population, and Sampling

From November 12, 2020 to December 15, 2020, we simultaneously conducted an online survey and an on-site survey among adult residents living in Mainland China. The cross-sectional anonymous online survey was done on the largest Chinese online survey platform, Wen Juan Xing (Changsha Ranxing Information Technology Co., Ltd., Hunan, China). The sample database of Wen Juan Xing consists of over 2.6 million respondents with confirmed personal information. This allows us to collect authentic and representative samples. A total of 2,013 respondents were enrolled in the online survey after excluding incomplete and invalid questionnaires.

Meanwhile, we conducted a cross-sectional on-site survey in five provinces/municipalities in China, namely, Guangdong, Zhejiang, Hubei, Jilin, and Chongqing. A random sampling method was adopted in the on-site survey, and a total of 4,316 valid questionnaires were collected after quality control and manual check procedures. Samples in the two cross-sectional surveys were merged for analysis. The two surveys were approved by Peking University Institutional Review Board (IRB00001052-20011).

Measures

The online and on-site questionnaires were designed according to a previous study on COVID-19 vaccine acceptance (27) and other studies estimated the acceptance of vaccination against emerging infectious diseases (28–31). Information collected in the questionnaires included (1) socio-demographic characteristics of respondents, such as age, gender, and living residence and (2) acceptance and attitude for COVID-19 vaccination. Most questions were closed-ended and treated as categorical variables. In this study, the primary outcome measure was the acceptance of the COVID-19 vaccination. Based on the question “If a COVID-19 vaccine is successfully developed and approved for using in the future, would you accept the vaccination,” respondents were classified into vaccine accepted group or the refused group in the surveys.

Statistical Analysis

Descriptive statistics were applied to present some baseline characteristics of respondents enrolled in the surveys, in which categorical variables were displayed as frequencies and proportions. The monetary amount used in the questionnaire was Chinese yuan (CNY). We provided an equivalence in the US Dollars (USD) at an exchange rate of 6.52 yuan per dollar in 2021. To identify the influencing factors of vaccine acceptance for different types of surveys, multivariate logistic regressions were conducted between the online and on-site sample, also for the two surveys respectively. The odds ratio (OR) and 95% CI were calculated and reported.

The propensity score matching (PSM) method was used to match “post-randomization” to directly compare the vaccination acceptance in online and on-site surveys. In the present study, the two samples varied in many socio-economic characteristics, therefore, it is difficult to distinguish the impact of survey type from other factors. The PSM proposed by Rosenbaum and Rubin can create a quasi-random process to correct for selection bias (32), providing an alternative for estimating treatment effects when systematic differences between groups are not random (33). The logit model is recommended to estimate the propensity score (34), which was also adopted in the present study by matching 12 covariates, such as age distribution, gender, region, living residence, marital status, education level, employment, household income, household size (the existence of the elderly was separately analyzed), and health status (the prevalence chronic disease was separately analyzed). The common ranges of propensity score and the standardized bias were graphed. A two-sided p-value below 0.05 was considered statistically significant in the present study. All data were analyzed using STATA, version 14.0 (Stata Corp, College Station, TX, USA) with two-tailed tests.

Results

Respondent Characteristics

Table 1 presents the basic characteristics of respondents. In the online survey, a total of 2,013 respondents completed the online questionnaires. Respondents were located in all 31 provincial administrative regions of Mainland China. The majority of them (98.3%) were under 60 years old, although we had tried to include as many participants aged above 60 years old as we could. Among the respondents, 49.0% were female, 72.3% were married, 85.1% were employed, and 65.1% resided in eastern China. For educational level, 34.6% of respondents had a high school and below degree, and 65.4% had an associate or bachelor degree and above. Total 68.0% of the respondents thought their health status were good or very good, and 87.4% of the respondents did not report any chronic diseases. Around half of the respondents (48.6%) had an annual family income of CNY 50,000–150,000 (USD 7,670–23,010). The respondents mainly (60.9%) lived in a family of 3–4 members and 55.0% of them did not live with the elderly.

In the on-site survey, 4,316 respondents were located in five provincial administrative regions in China, and around half of them (43.4%) were above 60 years old, showing a sample with more senior citizens. Among all respondents, 52.1% were employed, 46.5% lived in central China, and 64.2% lived in urban areas. The majority (69.2%) of respondents had a high school and below degree. Most of them (95.0%) thought their health status was good or very good, and 67.7% did not report any chronic diseases. Most of the respondents (68.5%) had an annual family income of less than CNY 100,000 (USD 15,340). Around one-third of the respondents lived in a family with 1–2 members (31.8%), and 58.2% of the respondents lived with the elderly.

Comparison of Acceptance of COVID-19 Vaccination Between the Two Surveys

Among the 2,013 respondents in the online survey, the proportion of general respondents who accepted COVID-19 vaccination was 90.0%, higher than that of the on-site survey (82.1%). Further comparing demographic characteristics of those who accepted the vaccination, differences were observed as 88.5% of online respondents who accepted to be vaccinated were below 60 years old, but the proportion was 48.2% in the on-site survey. Total 58.4% of online respondents who accepted were located in eastern China, while there were 24.5% in eastern China and 37.7% in central China in the on-site survey. Total 77.6% of online respondents who accepted were employed, but in the on-site survey, 44.5% respondents were employed. The results showed the differences in vaccination acceptance across differentiated demographic locations and socioeconomic characteristics.

Influencing Factors of Vaccine Acceptance Between the Two Surveys

Since the on-site results showed a lower level of acceptance of vaccination, the multivariate logistic regression was then performed between the online survey group and the on-site survey group to identify influencing factors of vaccination acceptance. Data from the online and on-site samples were pooled together in the logistic regression. We also conducted logistic regressions for the online and on-site samples, respectively. The results of regression models are presented in Table 2. The regression of pooled data showed that compared with online respondents, those in the on-site survey had significantly lower vaccine acceptance (OR: 0.49, 95% CI: 0.39–0.63). In addition, those aged above 60 years (OR: 0.73, 95% CI: 0.54–0.98) and considered their health status as normal or poor (OR: 0.58, 95% CI: 0.44–0.75) were less intended to accept vaccination. In contrast, those located in eastern (OR: 1.27, 95% CI: 1.07–1.52) or western China (OR: 2.10, 95% CI: 1.68–2.62), having a bachelor's degree (OR: 1.35, 95% CI: 1.07–1.70) or master's degree or above (OR: 1.78, 95% CI: 0.95, 3.31), having the elderly at home (OR: 1.37, 95% CI: 1.16–1.62), and being unemployed (OR: 1.45, 95% CI: 1.22–1.73) were more likely to accept vaccination. In the logistic regression for the online and on-site sample alone, the household income became a significant influencing factor of vaccination acceptance. In the online survey regression, higher household income ranging from CNY 150,000 to 200,000 (OR: 1.806, 95% CI: 0.971–3.359) and from CNY 200,000 to 300,000 (OR: 2.099, 95% CI: 1.025–4.298) led to stronger intention to be vaccinated. However, in the on-site survey regression results, those with household income from CNY 150,000 to 200,000 (OR: 0.562, 95% CI: 0.398, 0.794) were less intended to be vaccinated.

The Adjusted Acceptance and Influencing Factors Under PSM

Table 3 presents the adjusted results generated from the PSM using the nearest neighbor matching method among respondents in the online and on-site surveys. Before matching these two samples, the unmatched vaccine acceptance was 90.0% for the online survey and 82.1% for the on-site survey, showing a difference of 7.9%. In the PSM analysis, the individuals in the control group (the on-site sample) with the smallest difference in propensity score from those in the treatment group (the online sample) were compared. Controlled individuals were identified according to the information of treated individuals, and all treated individuals were paired successfully, so their information could be fully used.

After conducting PSM, the vaccination acceptance of the on-site survey was declined to 78.6%, with the difference increasing to 11.4% compared with the online survey. The average treatment effect for the treated (ATT) value was significant at a 5% significance level, indicating a difference in vaccination acceptance between the two surveys. Table 4 and Figure 1 show the common range of observed values under PSM. Of the 6,307 observations, 1,182 were not within the common range (off support), and the remaining 5,125 were within the common range (on support). Table 5 demonstrates the Propensity Score Testing (PSTEST) results, which examined whether the matched results could balance the differences of the values and checked whether there was a significant difference in the matched covariates between the two survey groups. For respondents who aged from 31 to 60 years, obtained high school or master and above degrees, being with annual household income more than CNY 50,000, with household size more than 2, not living with the elderly at home, with normal or poor health status, and without any chronic diseases, significant differences existed among these factors between the online and on-site survey groups. Figure 2 denotes the standardized bias across the covariates according to Table 4.

Figure 1

Figure 2

Discussion

To examine the public acceptance of COVID-19 vaccination in China and its differences under two survey methodologies, online and on-site surveys were conducted simultaneously during the well-contained phase of the pandemic (before the approval of COVID-19 vaccines). The present study revealed that the acceptance rate of COVID-19 vaccination among Chinese adults was as high as 90.0% in the online survey, demonstrating a minor reduction compared with the rate in the severe epidemic phase (91.9% in March 2020) (9). Meanwhile, the acceptance rate of vaccination was lower at 82.1% in the on-site survey. The difference suggested that even in the same phase of the pandemic using the same questionnaire questions, different survey methods could generate varied acceptance rates of vaccination. To interpret the difference in vaccination acceptance between the two surveys, several influencing factors should be considered, such as the demographic characteristics and self-perception of health status. According to the PSM results, the matched results indicated that the on-site acceptance rate declined to 78.6%, with an additional 3.5% in the acceptance gap compared to the online survey. A systematic difference in major outcomes and influencing factors indicated that different types of surveys could address discrepant results from the differentiated sampling and investigation measures.

Since survey techniques are widely used to explore human behaviors (35), they are frequently adopted in social and psychological research studies. By comparison, online surveys could be rapidly deployed and completed by respondents, particularly when disseminated via social media, web-based platforms, or where an incentive is offered for completion (36). Minimum cost would be incurred in online surveys because the questionnaire delivery and response can be completed automatically, reducing the payments to face-to-face investigators. Besides, the online survey could make it easier to understand complex or lengthy questions, which also avoid the complex process of coding and data cleaning to decrease data entry errors (25, 36, 37). However, the lack of an on-site interviewer can be a disadvantage in closed-ended online surveys, as respondents could not be clarified of unfamiliar or ambiguous terms in questionnaires (36). Moreover, the accumulation of biased or non-representative responses is also a drawback of the online surveys as those who lacked Internet access would not be captured, such as the elderly, those with lower income, or reside in remote rural areas (37, 38). For example, in the present study, the vaccination acceptance was higher among those aged below 60 years old in the online survey, compared with the on-site survey. After applying the PSM analysis, we deduced that the lack of access to web-based platforms might result in a lower level of participation of the elderly, further causing the bias of higher vaccination acceptance in the online survey.

During the pandemic in most countries, although the online survey is one of the few choices to conduct public investigations given the limitation of social distancing, self-administered online questionnaires are actually not a very useful tool for approaching illiterate or non-literate populations or those who cannot proficiently use technologies. In contrast, on-site surveys could effectively assign the sample by different regions and populations based on the socioeconomic development and specific research purposes, supporting to collect more representative responses. The potential sample bias could also be addressed by conducting on-site surveys, which could decline the possibility of over-representation to a particular viewpoint and the survey fraud (e.g., duplicate responses, false information, or deliberately exaggerated responses) (37, 39, 40). In the present study, PSM analysis results showed that the on-site vaccination acceptance rate was dropped to 78.6% after matching, with significant biases in factors, such as age, household income, household size, and health status. It indicates that systematic differences caused by the online investigation approach actually existed in evaluating public acceptance of COVID-19 vaccination, and its influencing factors were converged to a higher level of acceptance. The virtual-high results might further mislead the authority that most Chinese citizens were highly willing to get vaccinated, which is contrary to the fact that the Chinese government still needs to raise public awareness and acceptance toward COVID-19 vaccination.

Currently, it is not convenient to conduct on-site surveys in other countries, so most research groups implemented online surveys rather than on-site ones to collect public acceptance of COVID-19 vaccination. Since we conducted the surveys during the well-contained phase of the pandemic in China where the pandemic was effectively controlled, we were able to implement an on-site survey across five provinces/municipalities and compare the online and on-site results in terms of vaccine acceptance. Although PSM analysis may not completely solve the endogenous problem, it can effectively alleviate the deviation caused by self-selection. It can be illustrated from the survey and PSM results that the systematic difference of vaccine acceptance existed in the two types of surveys, but the difference gap was relatively small (90.0% compared with 78.6%).

Compared with other countries, even in the well-contained phase, the acceptance of vaccination in China remained higher. Studies reported that public acceptance of COVID-19 vaccination ranged from 62 to 80% in some European countries, among which Denmark and the UK had the highest acceptance (80%), while France (58.9–62%) and Italy (59%) had the lowest. While in Asian countries, the acceptance rates were relatively higher, as 79.8% in South Korea, 67–93.3% in Indonesia, and 94.3% in Malaysia (10–13, 41–45). Based on our findings, since an online survey might address the relatively higher acceptance rate of COVID-19 vaccination, it is possible that the real public acceptance of COVID-19 vaccination in those countries is lower than the reported survey results. Therefore, specific population groups are suggested to be considered in the online surveys to improve the representativeness of the study sample, by either combing with on-site interviews to reach specific population groups, or adjusting the outcomes through statistical approaches, such as PSM (37). Additionally, online questionnaires are recommended to improve the explicit delivery with some interpretations for key questions (36, 37, 40) and set duplicated questions to check for internal consistency. Encouraging the participants by giving specific incentives could be effective to improve the quality of online surveys, especially when the targeted participants have no or low level of income.

For limitations of this study, firstly, our study did not stratify the sample size by urban and rural areas of western China in the on-site survey, while geographic locations will inevitably affect the vaccination acceptance. It is due to that in western China, we conducted the on-site survey in Chongqing, most part of which is urban area with higher vaccination acceptance compared with rural areas. However, in central and eastern China, we conducted the survey in both urban and rural areas. This could explain why the acceptance rate in the western area was higher in the present study. Secondly, we did not conduct on-site surveys in all 31 provincial administrative regions as done in an online survey, but we collected samples from eastern, central, and western China in both the surveys, which help to minimize the deviations caused by geographical location differences. Thirdly, there was a gap in a sample size of the online (n = 2,013) and on-site survey (n = 4,316), but we have tried our best to collect data during the exact same time period for the two surveys, with sample size for each was over 2,000. This was due to some objective restrictions in the web-based survey method. According to Couper, web-based surveys present challenges particularly in sampling, coverage, non-response, and measurement errors (46). Without offering a paper-based questionnaire, a small but potentially important group of populations who did not have access to the Internet or did not receive the notification of online questionnaires would likely be missing with potentially biased estimates in the results (47). Fourthly, based on the fact that there were more senior participants in the on-site survey compared with the online survey, the selection bias within the results would possibly mislead the analysis for public acceptance of vaccination. To minimize the bias, we adjusted the results by applying the PSM methodology to estimate treatment effects when the systematic differences between the two surveys were not random, and all treated individuals were paired successfully.

Conclusion

This study simultaneously conducted the online and on-site survey toward the acceptance of COVID-19 vaccination among the Chinese population during the well-contained phase of the pandemic, which was the first large-scale on-site survey of COVID-19 vaccination acceptance and the first study to compare the simultaneous online and on-site results, both in China and in the world. From our study, a 7.9% gap was reported between the results of online and on-site surveys in a public acceptance rate of COVID-19 vaccination, and the gap became larger to 11.4% after PSM adjusting. Besides, multivariate logistic regression was conducted with the variables of age-distribution, region, education, employment status, household income, household size, and health status, which would affect vaccination acceptance. During the pandemic, it is not convenient to conduct on-site surveys in other countries, so most research groups implemented online surveys rather than on-site ones. Therefore, based on the findings of our study, statistical approaches could help to reduce the biases and enhance the rigor of online survey results to make them closer to the results generated from the on-site field survey. However, although statistical approaches could be applied to adjust the online survey results, we still recommend global researchers to combine online surveys with some small-scaled on-site surveys to ensure the capture of valid responsiveness and appropriate sample stratification.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

• 1.

  HuangCWangYLiXRenLZhaoJHuYet al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. (2020) 395:497–506. 10.1016/S0140-6736(20)30183-5

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 2.

  WHO. Coronavirus Disease (COVID-19) Dashboard. Available online at: https://covid19.who.int/ (accessed June 20, 2021).

  • Google Scholar

• 3.

  WHO. Overview of the COVID-19 Pandemic. Available online at: https://covid19.who.int/docs/default-source/coronaviruse/risk-comms-updates/epi/win-update40 (accessed April 11, 2021).

  • Google Scholar

• 4.

  WHO. WHO COVID-19 Weekly Operational Update. Available online at: https://covid19.who.int/docs/default-source (accessed April 21, 2021).

  • Google Scholar

• 5.

  MaringeCSpicerJMorrisMPurushothamANolteESullivanRet al. The impact of the COVID-19 pandemic on cancer deaths due to delays in diagnosis in England, UK: a national, population-based, modelling study. Lancet Oncol. (2020) 21:1023–34. 10.1016/S1470-2045(20)30388-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 6.

  WHO. Draft Landscape of COVID-19 Candidate Vaccines. Available online at: https://www.who.int/publications/m/item/draft-landscape-of-covid-19-candidate-vaccines (accessed April 11, 2021).

  • Google Scholar

• 7.

  WHO. Updates on COVID-19 Vaccines Development. Available online at: https://who.int/doc/default-sourc/coronaviruse/risk-comms-updates (accessed April 8, 2021).

  • Google Scholar

• 8.

  WHO. Draft Landscape and Tracker of COVID-19 Candidate Vaccines. Available online at: https://www.who.int/publications/m/item/draft-landscape-of-covid-19-candidate-vaccines (accessed April 8, 2021).

  • Google Scholar

• 9.

  WangJJingRLaiXZhangHLyuYKnollMDet al. Acceptance of COVID-19 vaccination during the COVID-19 pandemic in China. Vaccines. (2020) 8:482. 10.3390/vaccines8030482

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 10.

  LazarusJVRatzanSCPalayewAGostinLOLarsonHJRabinKet al. A global survey of potential acceptance of a COVID-19 vaccine. Nat Med. (2020) 27:225–28. 10.1101/2020.08.23.20180307

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 11.

  LinYHuZZhaoQAliasHDanaeeMWongLP. Understanding COVID-19 vaccine demand and hesitancy: a nationwide online survey in China. PLoS Negl Trop Dis. (2020) 14:e0008961. 10.1371/journal.pntd.0008961

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 12.

  Neumann-BhmeSSabatI. Now, We Have It. Will We Use It? New Results From ECOS on the Willingness to be Vaccinated Against COVID-19.Hamburg: Improving Quality of Care in Europe (IQCE).(2021). 10.1007/s10198-020-01208-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 13.

  DetocMBruelSFrappePTardyBBotelho-NeversEGagneux-BrunonA. Intention to participate in a COVID-19 vaccine clinical trial and to get vaccinated against COVID-19 in France during the pandemic. Vaccine. (2020) 38:7002–6. 10.1016/j.vaccine.2020.09.041

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 14.

  European Centre for Disease Prevention and Control. COVID-19 Situation Update Worldwide, as of Week 40, updated 14 October 2021. Available online at: https://www.ecdc.europa.eu/en/geographical-distribution-2019-ncov-cases (accessed October 15, 2021).

  • Google Scholar

• 15.

  Onyeaka H Anumudu CK Al-Sharify ZT and Egele-Godswill E Mbaegbu P. COVID-19 pandemic: a review of the global lockdown its far-reaching effects. Sci Prog. (2021) 104:368504211019854. 10.1177/00368504211019854

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 16.

  DiDomenico LPullanoGSabbatiniCEBoëllePYColizzaV. Impact of lockdown on COVID-19 epidemic in Île-de-France and possible exit strategies. BMC Med. (2020) 18:240. 10.1186/s12916-020-01698-4

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 17.

  JoffeAR. COVID-19: rethinking the lockdown groupthink. Front Public Health. (2021) 9:625778. 10.3389/fpubh.2021.625778

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 18.

  PachettiMMariniBGiudiciFBenedettiFAngelettiSCiccozziMet al. Impact of lockdown on COVID-19 case fatality rate and viral mutations spread in 7 countries in Europe and North America. J Transl Med. (2020) 18:338. 10.1186/s12967-020-02501-x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 19.

  ZhangSWangZChangRWangHXuCYuXet al. COVID-19 containment: China provides important lessons for global response. Front Med. (2020) 14:215–9. 10.1007/s11684-020-0766-9

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 20.

  XuTLAoMYZhouXZhuWFNieHYFangJHet al. China's practice to prevent and control COVID-19 in the context of large population movement. Infect Dis Poverty. (2020). 9:115. 10.1186/s40249-020-00716-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 21.

  ChenSYangJYangWWangCBärnighausenT. COVID-19 control in China during mass population movements at New Year. Lancet. (2020) 395:764–6. 10.1016/S0140-6736(20)30421-9

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 22.

  LiuWYueX GTchounwouP B. Response to the COVID-19 epidemic: the chinese experience and implications for other countries. Int J Environ Res Public Health. (2020) 17:2304. 10.3390/ijerph17072304

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 23.

  BurkiT. China's successful control of COVID-19. Lancet Infect Dis. (2020) 20:1240–1. 10.1016/S1473-3099(20)30800-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 24.

  EysenbachGWyattJ. Using the internet for surveys and health research. J Med Internet Res. (2002) 4:E13. 10.2196/jmir.4.2.e13

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 25.

  CallegaroMManfredaKLVehovarV. Web Survey MethodologyLos Angeles: Sage Publications Ltd (2015).

  • Google Scholar

• 26.

  SueVMRitterLA. Conducting Online Surveys. Los Angeles: Sage (2012). 10.4135/9781506335186

  • CrossRef
  • Google Scholar

• 27.

  WangJLyuYZhangHJingRLaiXFengHet al. Willingness to pay and financing preferences for COVID-19 vaccination in China. Vaccine. (2021) 39:1968–76. 10.1016/j.vaccine.2021.02.060

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 28.

  MadduxJERogersRW. Protection motivation and self-efficacy: a revised theory of fear appeals and attitude change. Exp Soc Psychol. (1983) 19:469–79. 10.1016/0022-1031(83)90023-9

  • CrossRef
  • Google Scholar

• 29.

  MaurerJHarrisKMParkerALurieN. Does receipt of seasonal influenza vaccine predict intention to receive novel H1N1 vaccine: evidence from a nationally representative survey of U.S. adults. Vaccine. (2009) 27:5732–4. 10.1016/j.vaccine.2009.07.080

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 30.

  EastwoodKDurrheimDNJonesAButlerM. Acceptance of pandemic (H1N1) 2009 influenza vaccination by the Australian public. Med J Aust. (2010) 192:33–6. 10.5694/j.1326-5377.2010.tb03399.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 31.

  SypsaVLivaniosTPsichogiouMMallioriMTsiodrasSNikolakopoulosIet al. Public perceptions in relation to intention to receive pandemic influenza vaccination in a random population sample: evidence from a cross-sectional telephone survey. Euro Surveill. (2009) 14:19437. 10.2807/ese.14.49.19437-en

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 32.

  RosenbaumPRRubinDB. The central role of the propensity score in observational studies for causal effects. Biometrika. (1983) 70:41–55. 10.1093/biomet/70.1.41

  • CrossRef
  • Google Scholar

• 33.

  ThoemmesFJKimES. A systematic review of propensity score methods in the social sciences. Multivariate Behav Res. (2011) 46:90–118. 10.1080/00273171.2011.540475

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 34.

  RosenbaumPRubinDB. Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. Am Stat. (1985) 39:33–8. 10.1080/00031305.1985.10479383

  • CrossRef
  • Google Scholar

• 35.

  PontoJ. Understanding and evaluating survey research. J Adv Practition Oncol. (2015) 6:168. 10.6004/jadpro.2015.6.2.9

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 36.

  BallHL. Conducting online surveys. J Hum Lact. (2019) 35:413–7. 10.1177/0890334419848734

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 37.

  AlessiE. Conducting an internet-based survey: benefits, pitfalls, and lessons learned. Soc Work Res. (2010) 34:122–8. 10.1093/swr/34.2.122

  • CrossRef
  • Google Scholar

• 38.

  AndrewsDNonneckeBPreeceJ. Electronic survey methodology: a case study in reaching hard-to-involve internet users. Int J Hum Comput Interact. (2003) 16:185–210. 10.1207/S15327590IJHC1602_04

  • CrossRef
  • Google Scholar

• 39.

  BohannonJ. Survey fraud test sparks battle. Science. (2016) 351:1014. 10.1126/science.351.6277.1014

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 40.

  LinCTuPBeitschL M. Confidence and receptivity for COVID-19 vaccines: a rapid systematic review. Vaccines. (2021) 9:16. 10.3390/vaccines9010016

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 41.

  PalamenghiLBarelloSBocciaSGraffignaG. Mistrust in biomedical research and vaccine hesitancy: the forefront challenge in the battle against COVID-19 in Italy. Eur J Epidemiol. (2020) 35:785–8. 10.1007/s10654-020-00675-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 42.

  WongLPAliasHWongPFLeeHYAbuBakarS. The use of the health belief model to assess predictors of intent to receive the COVID-19 vaccine and willingness to pay. Hum Vaccines Immunother. (2020) 16:2204–14. 10.1080/21645515.2020.1790279

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 43.

  HarapanHWagnerALYufikaAWinardiWAnwarSGanAKet al. Acceptance of a COVID-19 vaccine in Southeast Asia: a cross-sectional study in Indonesia. Front Public Health. (2020) 8:381. 10.3389/fpubh.2020.00381

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 44.

  SzilagyiPGThomasKShahMDVizuetaNCuiYVangalaSet al. National trends in the US public's likelihood of getting a COVID-19 vaccine-April 1 to December 8, 2020. JAMA. (2021) 325:396–8. 10.1001/jama.2020.26419

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 45.

  CouperMP. Review: web surveys: a review of issues and approaches. Public Opin Q. (2000) 64:464–94. 10.1086/318641

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 46.

  KelfveSKiviMJohanssonBLindwallM. Going web or staying paper? The use of web-surveys among older people. BMC Med Res Methodol. (2020) 20:252. 10.1186/s12874-020-01138-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 47.

  BernardHR. Research Methods in Anthropology: Qualitative and Quantitative Approaches. 5th ed. New York, NY: Altamira Press (2011).

  • Google Scholar