Published on in Vol 8, No 3 (2022): March

Preprints (earlier versions) of this paper are available at, first published .
Official Websites Providing Information on COVID-19 Vaccination: Readability and Content Analysis

Official Websites Providing Information on COVID-19 Vaccination: Readability and Content Analysis

Official Websites Providing Information on COVID-19 Vaccination: Readability and Content Analysis

Original Paper

1Oral and Maxillofacial Radiology, Applied Oral Sciences and Community Dental Care, Faculty of Dentistry, The University of Hong Kong, Hong Kong, China

2Ludwig Boltzmann Institute for Digital Health and Patient Safety, Vienna, Austria

3Division of Pediatric Nephrology and Gastroenterology, Department of Pediatrics and Adolescent Medicine, Comprehensive Center for Pediatrics, Medical University of Vienna, Vienna, Austria

4Department of Translational Stem Cell Biology, Research Institute of the Medical University of Varna, Varna, Bulgaria

5Medical University of Innsbruck, Innsbruck, Austria

6Department of Computer Science, University of Innsbruck, Innsbruck, Austria

7Graz University of Technology, Graz, Austria

8Institute for Ethics and Law in Medicine, University of Vienna, Vienna, Austria

9Department of Anaesthesia, Intensive Care Medicine and Pain Medicine, Medical University of Vienna, Vienna, Austria

10Institute of Genetics and Animal Biotechnology, Polish Academy of Sciences, Jastrzebiec, Poland

Corresponding Author:

Atanas G Atanasov, PhD

Ludwig Boltzmann Institute for Digital Health and Patient Safety

Spitalgasse 23

Vienna, 1090


Phone: 43 664 1929 852


Background: Online information on COVID-19 vaccination may influence people’s perception and willingness to be vaccinated. Official websites of vaccination programs have not been systematically assessed before.

Objective: This study aims to assess and compare the readability and content quality of web-based information on COVID-19 vaccination posted on official/governmental websites. Furthermore, the relationship between evaluated website parameters and country vaccination rates were calculated.

Methods: By referring to an open data set hosted at Our World in Data, the 58 countries/regions with the highest total vaccination count as of July 8, 2021, were identified. Together with the websites from the World Health Organization and European Union, a total of 60 vaccination campaign websites were targeted. The “frequently asked questions” or “questions and answers” section of the websites were evaluated in terms of readability (Flesch Reading Ease score and Flesch-Kincaid Grade Level), quality (Health On the Net Foundation code [HONcode] certification and Quality Evaluation Scoring Tool), and content stating vaccination duration of protection and potential side effects.

Results: In terms of readability, the Flesch Reading Ease score of the vaccination frequently asked questions websites ranged between 11.2 and 69.5, with a mean of 40.9 (SD 13.2). Meanwhile, the Flesch-Kincaid Grade Level ranged between 6.5 and 17.6, with a mean of 12.1 (SD 2.8). In terms of quality, only 2 websites were HONcode certified, and the Quality Evaluation Scoring Tool score of the websites ranged between 7 and 20, with a mean of 15.3 (SD 3.1). Half of the websites (25/50) did not present a publication date or date of the last update. Regarding the duration of protection offered by the vaccines, 46% (23/50) of the websites stated that they do not know, and another 40% (20/50) did not address it. Five side effects of the vaccinations were most frequently mentioned, namely, fever/chill (41/50, 82%), various injection site discomfort events (eg, swelling, redness, or pain; 39/50, 78%), headache (36/50, 72%), fatigue (33/50, 66%), and muscle/joint pain (31/50, 62%).

Conclusions: In general, the content quality of most of the evaluated websites was good, but HONcode certification should be considered, content should be written in a more readable manner, and a publication date or date of the last update should be presented.

JMIR Public Health Surveill 2022;8(3):e34003



The COVID-19 pandemic has affected the global population since the end of 2019 and has no signs of resolution as of late 2021. Its fatality rate varied by population and viral strains, with earliest rates reported to be 2.3% in China and 7.2% in Italy [1]. A recent meta-analysis computed an overall infection fatality rate of 0.68% [2]. The COVID-19 morbidity has incurred substantial economic burden, and a study examining the matter estimated that the mean direct medical costs (cost incurred by patient care in the hospital; eg, bed service, consultation, nursing, imaging, medicine, and laboratory) were US $3755 and indirect costs (including income loss due to premature death and productivity loss due to hospitalization and during recovery) were US $11,634 per person [3]. Another study reported that patients with COVID-19 stayed in the hospital for 5 days on average, and the incurred median hospital costs were US $12,046 [4]. In short, the COVID-19 pandemic has caused a heavy burden on society and public health on a global scale. Being a powerful tool to counteract infection, currently available vaccines against COVID-19 may substantially aid mitigation of this pandemic.

Meanwhile, vaccination against COVID-19 became possible at the end of 2020. In general, vaccination is able to provide both health benefits for people, as well as to save medical care costs; reduce productivity loss; improve outcomes in unvaccinated community members (eg, through herd effects); and maintain economic, social, and political stability [5]. Vaccination may by some people even be perceived as a social contract that considers social welfare and moral obligation beyond self-interest, as a recent study found that the more compliant individuals regarding vaccination showed less generosity toward those who were not vaccinated [6].

On this background, vaccine hesitancy poses an issue for maximizing the population that could be vaccinated. It was found that one-third of the United Kingdom’s and Ireland’s population display COVID-19 vaccine hesitancy or even resistance [7]. Concerns about side effects and vaccine efficacy were two main reasons for not getting the COVID-19 vaccine [8]. Misinformation might worsen this situation, and in this regard, the provision of official COVID-19–related information in a comprehensible way could prevent people from seeking information or misinformation from nontrustworthy websites [9]. For example, YouTube has boosted the search rankings of provaccine videos, but viewers directed to antivaccine videos were likely to be exposed to additional links to antivaccine videos [10]. Therefore, governmental or official websites should serve as authoritative sources of COVID-19–related information. Readability and quality metrics of these websites are important, as the internet may eliminate barriers in the access to health information and hence eliminate misinformation for the public if the presented materials are of good quality and can be easily comprehended by the majority of the population [11]. Difficulty reading vaccine information may influence attitudes toward acceptance of or hesitancy to take vaccinations [12]. Previously, it was found that campaign websites could boost vaccine coverage [13]. The exact reasons were unclear, and it could have two implications. First, it could be that better websites would lead to higher vaccine coverage. Second, it could also be that places with lower vaccine coverage would put more effort to optimize their websites to promote/increase vaccine coverage. Regardless, it would be relevant to evaluate the relationship between website metrics and vaccination data.

This study aims to assess the readability and content quality of official COVID-19 vaccination campaign websites worldwide to reveal if their contents were easily understandable and mentioned the side effects and duration of vaccine protection, which are important factors in determining the support for getting vaccinated [14].

Data Source and Search Strategy

The open and actively updated data set named Coronavirus (COVID-19) Vaccinations [15], hosted by the Our World in Data database, was accessed on July 8, 2021. There were 58 countries/regions with their total vaccination count exceeding 4 million doses. For these 58 countries/regions the total vaccination count and vaccination dose per 100 people were noted. The governmental or official vaccination website was searched via Google with the phrases “official,” “COVID,” and “vaccine.” After entering these three terms, the first 10 result pages were examined to identify the relevant official vaccination website. The “frequently asked questions” (FAQs) or “questions and answers” (Q&As) section was identified and analyzed. This particular section was selected assuming that this would be the page where citizens sought for official information in the first place. Together with the websites of the World Health Organization (WHO) and European Union, we attempted to collect information from a total of 60 websites. As many European countries are in the European Union, the public living in Europe may look for information not only from their own country’s website but also from the EU website. Similarly, the WHO is the global health authority, and many people may look for information from its website as well. Therefore, websites from the WHO and European Union were included in this study. If there were multiple websites for one country (eg, vaccination websites from different governmental departments), then the one with FAQs targeting the public was selected. If such a section was not available from any of the websites, then the country would be coded as “cannot locate the vaccination FAQ website” (n=10; see Figure 1). The total vaccination count and vaccination dose per 100 people would be assessed with the website metrics to reveal if a better prepared website would relate to better vaccination coverage.

Figure 1. Flowchart of the selection of governmental or official COVID-19 vaccination websites for evaluation. FAQ: frequently asked question; Q&A: question and answer.
View this figure

The website identification was performed by four authors (AWKY, EK, EDP, and AGA). The data extraction was performed by AWKY and double-checked by AGA. Disagreements in these procedures were solved by discussion and consensus reached between AWKY and AGA.

Readability Assessment

For readability assessment (limited to websites with English versions only), the first 5 answers from the FAQs or Q&As page were transferred to Office Word software (Microsoft Corporation). The first 5 answers were used for evaluation, as online health information seekers rarely go beyond the first page of a search [16], and we reasoned that 5 FAQs would occupy a similar page space. The Flesch Reading Ease score and Flesch-Kincaid Grade Level statistic [17] were computed automatically by its built-in Spelling and Grammar function. A higher Flesch Reading Ease score indicates the text is more readable: reading difficulty is considered very easy if the score approaches 100, whereas the score range of 60 to 70 is considered standard [18,19]. In contrast, the value for the Flesch-Kincaid Grade Level indicates the grade level required for a reader to understand the text [17]. The American Medical Association and National Institutes of Health recommended that patient information materials should be written in the third to seventh grade level [20,21]. In the United States, the third grade means the third school year after kindergarten (third year of primary school) when students are aged around 8 to 9 years. Similarly, seventh grade means the seventh school year after kindergarten when students are aged around 12 to 13 years. The readability of websites without the English version were not evaluated, but they were still evaluated for the subsequent analyses (quality assessment and listing of protection duration and side effects) following Google Translation to English.

Quality Assessment

Content quality was first assessed by the Health On the Net Foundation code (HONcode) certification status (ie, whether a website bore the HONcode badge). The Health on the Net Foundation is affiliated with the United Nations and issues the HONcode for health care websites that filed an application and met the required standard [22]. A dedicated web browser toolbar can be installed [23] to identify the certification status of a website, as used by previous researchers [24]. Having a HONcode certification is good, as it indicates the website reaches a standard of offering quality health information.

As this HONcode only gave a binarized certification status (certified or not) to the websites, we also used the Quality Evaluation Scoring Tool (QUEST) to evaluate the FAQ sections in seven aspects, namely, authorship, attribution to sources/references, type of study attributed to, conflict of interest, currency of presented information, complementarity of the patient-doctor relationship (support or no support of the relationship), and tone [25]. Readers can refer to the second figure by Robillard et al [25], published with open-access, for the complete description of the coding and scoring of QUEST. There is no cutoff score for QUEST, but the original research team reported that a score ≤11 indicated the worst scores from a survey of 290 online articles [26].

Listing of Protection Duration and Side Effects

We recorded the number of Q&As from each website. The stated protection duration and the side effects of vaccinations listed on the websites were also recorded.

Data Collection

It was not possible to collect all data within a single day. Data were collected since the day of the database search (July 8, 2021). On the last day of data collection (September 29, 2021), the publication dates and dates of the last update from all websites were collected to ensure a fairer evaluation of recency.

Statistical Analysis

The Pearson correlation test was conducted to evaluate if the two readability metrics, QUEST score, and number of Q&As were correlated with the total vaccination count and vaccination dose per 100 people. Data from the WHO and European Union were not included in the correlation test or regional counts. Statistical tests were performed with SPSS (version 26.0; IBM Corp). Results were deemed significant if P<.05.

Overall Data Collection Situation

The 60 investigated vaccination FAQs websites are listed in Table 1. By the United Nations geoscheme, 22 of the 58 selected countries/regions were in Asia, 22 in Europe, 4 in North America, 7 in South America, 2 in Africa, and 1 in Oceania. There were 10 countries for which we could not locate a vaccination FAQs website. There were 20 countries that provided non-English websites only. Therefore, quality and content data were available from 50 websites, whereas readability data were available from 30 websites.

Table 1. The 60 investigated governmental or official COVID-19 vaccination websites as accessed on July 8, 2021 (listed in descending order of total vaccination count).
Country/regionTotal vaccination count (million), nVaccination dose per 100 people, nOfficial vaccination FAQsa website
United States331.799.2[29]
United Kingdom79.5117.2[32]
South Korea19.938.8[45]
Colombia19.438.2Cannot locate the vaccination FAQs website
Saudi Arabia19.154.8[47]
United Arab Emirates15.7159.0[50]
Hungary9.8101.9Cannot locate the vaccination FAQs website
Dominican Republic8.477.4Cannot locate the vaccination FAQs website
Cambodia8.148.7Cannot locate the vaccination FAQs website
Cuba6.859.7Cannot locate the vaccination FAQs website
Iran5.76.8Cannot locate the vaccination FAQs website
Serbia5.378.2Cannot locate the vaccination FAQs website
Ecuador4.827.2Cannot locate the vaccination FAQs website
Egypt4.54.4Cannot locate the vaccination FAQs website
Sri Lanka4.521.2Cannot locate the vaccination FAQs website
Hong Kong4.155.3[74]
World Health OrganizationN/AbN/A[75]
European CommissionN/AN/A[76]

aFAQ: frequently asked question.

bN/A: not applicable.

Readability Assessment

The Flesch Reading Ease score of the vaccination FAQs websites ranged between 11.2 and 69.5, with a mean of 40.9 (SD 13.2). The websites of Sweden and the United Kingdom had a score above the threshold of 60 (Figure 2). In particular, the most difficult website was from the Philippines, with a score of 11.2.

Figure 2. Readability assessment of the governmental or official COVID-19 vaccination websites. The recommended standards are >60 for Flesch Reading Ease score (blue bar) and <7 for Flesch-Kincaid Grade Level (orange line). WHO: World Health Organization.
View this figure

Meanwhile, the Flesch-Kincaid Grade Level of the websites ranged between 6.5 to 17.6, with a mean of 12.1 (SD 2.8). Only the website of Sweden was within the targeted range (Flesch-Kincaid Grade Level 6.5; Figure 2).

Quality Assessment

Only the websites from Switzerland and the WHO were HONcode certified. The QUEST score of the analyzed websites ranged between 7 and 20, with a mean of 15.3 (SD 3.1). Only 5 websites did not reach the recommended score of 11 (Figure 3). A detailed breakdown of the score revealed that only Sweden and Canada explicitly mentioned the authorship. None of the websites listed references to identifiable scientific studies. Besides, half of the websites (25/50) did not present a publication date or date of the last update. For the 25 websites that listed a date of the last update, all of them were updated in the year 2021 (n=12 in September, n=2 in August, n=3 in July, n=2 in June, n=3 in April, n=2 in February, and n=1 in January). Four of them also listed a publication date, ranging from October 23, 2020, to April 7, 2021.

Figure 3. Quality assessment of the governmental or official COVID-19 vaccination websites. The recommended level of the QUEST score is ≥11 (orange line). QUEST: Quality Evaluation Scoring Tool; WHO: World Health Organization.
View this figure

Listing of Protection Duration and Side Effects

The number of Q&As listed in the websites ranged from 6 to 150, with a mean of 36.5 (SD 35.1). Regarding the duration of protection offered by the vaccines, 46% (23/50) of the websites stated that they do not know, and another 40% (n=20) did not address this aspect. The remaining websites stated that the vaccine could provide protection for at least 6 months (n=2), 6 to 8 months (n=1), at least 9 months (n=1), at least 9 to 12 months (n=1), and at least 12 months (n=2). Five side effects were most frequently mentioned, namely, fever/chill (n=41, 82%), various injection site discomfort events (eg, swelling, redness, or pain; n=39, 78%), headache (n=36, 72%), fatigue (n=33, 66%), and muscle/joint pain (n=31, 62%). Other side effects were mentioned much less frequently (<28%). Figure 4 lists the frequency counts of mentioned side effects.

Figure 4. List of side effects caused by vaccination as mentioned by the governmental or official COVID-19 vaccination websites. CNS: central nervous system.
View this figure

No Relationship Between Vaccination and Website Metrics

Pearson correlation tests revealed that the Flesch Reading Ease score, Flesch-Kincaid Grade Level, QUEST score, and number of Q&As had no significant relationship with total vaccination count (P=.35, .29, .47, and .23, respectively) and vaccination dose per 100 people (P=.30, .44, .33, and .59, respectively).

Principal Findings

As information on COVID-19 vaccination might influence vaccination progress, we set out to assess governmental websites providing information on COVID-19 vaccination in 60 countries/regions and organizations, and finally identified and analyzed 50 websites that promote COVID-19 vaccination programs.

Readability remained an issue for most of the analyzed websites. Even though we analyzed the FAQs section of the websites, which are supposed to be reader friendly, all but 1 failed to meet the recommendation by the American Medical Association and National Institutes of Health of being easily understood by a seventh grader or below [20,21], and only 2 met the recommendation of a Flesch Reading Ease score of 60 to 70 [18,19]. One of the potential reasons could be that the information disseminated in these vaccination campaign websites must be accurate and explanatory to minimize any risks of legal consequences. Hence, the choice of words would be careful and explanatory clauses added to lengthen the sentences. For example, the Philippines website was as difficult to read as a standard auto insurance policy (see Flesch [77] for more examples). It listed a simple question “are there enough vaccines for filipinos” with an answer composed of 2 long sentences with many polysyllabic words:

The prioritization framework for COVID-19 vaccination based on the WHO Strategic Advisory Group of Experts (SAGE) on Immunization, together with the recommendations of independent bodies of experts including the Interim National Immunization Technical Advisory Group (iNITAG) and the Technical Advisory Group (TAG), was formulated due to the limited global supply of COVID-19 vaccine products. With the guidance of this principle, the Philippine National Deployment and Vaccination Plan formulated strategies and contingencies to ensure the equitable distribution of vaccine products for all Filipinos.

Certainly, readability and content quality are two different aspects valued by online health information seekers. HONcode certification was reported as an indicator correlated to the content accuracy of online health information [78]. In this context, it is interesting to observe that the surveyed vaccination websites did not present certifications. Since the Health On the Net Foundation is affiliated with the United Nations and its certification is globally recognized, more efforts should be made to get certified, especially as these vaccination websites are accessed by many people on a daily and global basis. However, most of them actually had a QUEST score of ≥11, meaning that their content quality was good. The most frequently listed side effects in the websites were consistent to the ones most frequently reported by people receiving the COVID-19 vaccines [79,80]. This consistency is beneficial, as information seekers will feel that the website is trustworthy and provides accurate descriptions of what to be expected if they want to be vaccinated. Although none of the websites listed references to identifiable scientific studies, it could be argued that the FAQs websites were primarily intended for provision of the most important and updated information; references might not be highly sought after by the information seekers in this scenario.

The majority of the websites either did not mention the duration of protection or stated that they did not know how long vaccination could protect citizens. Despite this particular missing (or unknown) piece of information, these 48 countries/regions still had the highest total vaccination counts. To get vaccinated may be influenced by many different considerations, wherefore readability, quality, and content of the vaccination websites were not the only potential influencing factors. Higher frequency of exposure to positive information about COVID-19 vaccination was associated with a higher intention to be vaccinated [81]. Meanwhile, public health authorities should also monitor social media continuously to identify newly developed antivaccine arguments and provide updated information to clear the doubts and debunk the myths [82]. These aspects should be carefully considered by the authorities as they update/improve the campaign websites. This survey found that only a small proportion of the websites were last updated in September 2021. It suggested that the rest of these websites should be updated more frequently to provide up-to-date information.

Initially, it was a bit surprising to find that there was no relationship between vaccination counts and website metrics. However, it could be explained by the fact that people could access vaccine information via many sources such as newspapers, magazines, YouTube, and Facebook. A previous study [83] revealed that the commonest reasons for agreeing to be vaccinated included “to protect themselves and others” (29%), “belief in vaccination and science” (16%), and “to help stop the virus spread” (15%), which could be intuitive and might not be affected by vaccination website readability.

This study found that only 2 of the 50 surveyed websites were HONcode certified. This was consistent to a recent survey on online misinformation about COVID-19, for which the certification rate was reported to be 1.8% (2/110) [84]. It was similarly reported that it was difficult to comprehend the privacy policies of COVID-19 contact tracing apps, with their reading level between 7th and 14th grade [85]. Considered together, the take-home message for the authoritative bodies and policy makers is to further improve the COVID-19 vaccination campaign items (eg, websites or apps) by making them more comprehensible and more trustworthy with certification.

This study surveyed the FAQs section of official vaccination websites. However, people may seek online information from other sources such as online discussion forums, news outlets, and blogs. Readability and content quality of such sources was outside the scope of this study and might provide an interesting topic for further research. Furthermore, some official vaccination websites did not have FAQs sections, and thus, no data were extracted. Meanwhile, the initial decision to target 60 websites/countries was relatively arbitrary. Therefore, readers should be aware that not all websites across all countries were included. For the non-English language websites, content were only analyzed after translation to English via Google Translate. Therefore, readability could not be assessed. Even for the websites that presented English materials, some biases or issues may still exist. In particular, because English could be a primary language for some countries but not the others or even for certain populations within a country instead of the entire population, bias might be introduced. Hence, the content could be written as a primary version, a translated version, or even provided by automated website translation beyond our notice. Instead of choosing the desired language by clicking a button, a country might provide FAQs sections in other languages at completely different websites not evaluated in this study. In addition, the potential reuse of content from EU/WHO websites by country websites was not assessed. Readers should be aware of these issues during the interpretation of the readability assessment results.


The following points could be concluded: the content quality was good, HONcode certification should be considered, content should be written in a more readable manner, and web pages should be updated more frequently to keep the information up to date. In short, COVID-19 vaccination FAQs websites basically provide good quality information, but more efforts should be paid to make them more readable and updated.

Conflicts of Interest

None declared.

  1. Onder G, Rezza G, Brusaferro S. Case-fatality rate and characteristics of patients dying in relation to COVID-19 in Italy. JAMA 2020 May 12;323(18):1775-1776. [CrossRef] [Medline]
  2. Meyerowitz-Katz G, Merone L. A systematic review and meta-analysis of published research data on COVID-19 infection fatality rates. Int J Infect Dis 2020 Dec;101:138-148 [FREE Full text] [CrossRef] [Medline]
  3. Ghaffari Darab M, Keshavarz K, Sadeghi E, Shahmohamadi J, Kavosi Z. The economic burden of coronavirus disease 2019 (COVID-19): evidence from Iran. BMC Health Serv Res 2021 Feb 11;21(1):132 [FREE Full text] [CrossRef] [Medline]
  4. Di Fusco M, Shea KM, Lin J, Nguyen JL, Angulo FJ, Benigno M, et al. Health outcomes and economic burden of hospitalized COVID-19 patients in the United States. J Med Econ 2021;24(1):308-317 [FREE Full text] [CrossRef] [Medline]
  5. Bärnighausen T, Bloom DE, Cafiero-Fonseca ET, O'Brien JC. Valuing vaccination. Proc Natl Acad Sci U S A 2014 Aug 26;111(34):12313-12319 [FREE Full text] [CrossRef] [Medline]
  6. Korn L, Böhm R, Meier NW, Betsch C. Vaccination as a social contract. Proc Natl Acad Sci U S A 2020 Jun 30;117(26):14890-14899 [FREE Full text] [CrossRef] [Medline]
  7. Murphy J, Vallières F, Bentall RP, Shevlin M, McBride O, Hartman TK, et al. Psychological characteristics associated with COVID-19 vaccine hesitancy and resistance in Ireland and the United Kingdom. Nat Commun 2021 Jan 04;12(1):29. [CrossRef] [Medline]
  8. Solís Arce JS, Warren SS, Meriggi NF, Scacco A, McMurry N, Voors M, et al. COVID-19 vaccine acceptance and hesitancy in low- and middle-income countries. Nat Med 2021 Aug;27(8):1385-1394 [FREE Full text] [CrossRef] [Medline]
  9. Kricorian K, Civen R, Equils O. COVID-19 vaccine hesitancy: misinformation and perceptions of vaccine safety. Hum Vaccin Immunother 2021 Jul 30:1-8. [CrossRef] [Medline]
  10. Tang L, Fujimoto K, Amith M, Cunningham R, Costantini RA, York F, et al. "Down the Rabbit Hole" of vaccine misinformation on YouTube: network exposure study. J Med Internet Res 2021 Jan 05;23(1):e23262 [FREE Full text] [CrossRef] [Medline]
  11. Berland GK, Elliott MN, Morales LS, Algazy JI, Kravitz RL, Broder MS, et al. Health information on the Internet: accessibility, quality, and readability in English and Spanish. JAMA 2001;285(20):2612-2621 [FREE Full text] [CrossRef] [Medline]
  12. Okuhara T, Ishikawa H, Ueno H, Okada H, Kato M, Kiuchi T. Readability assessment of vaccine information: a systematic review for addressing vaccine hesitancy. Patient Educ Couns 2022 Feb;105(2):331-338. [CrossRef] [Medline]
  13. Odone A, Ferrari A, Spagnoli F, Visciarelli S, Shefer A, Pasquarella C, et al. Effectiveness of interventions that apply new media to improve vaccine uptake and vaccine coverage. Hum Vaccin Immunother 2015;11(1):72-82 [FREE Full text] [CrossRef] [Medline]
  14. Kreps S, Prasad S, Brownstein JS, Hswen Y, Garibaldi BT, Zhang B, et al. Factors associated with US adults' likelihood of accepting COVID-19 vaccination. JAMA Netw Open 2020 Oct 01;3(10):e2025594 [FREE Full text] [CrossRef] [Medline]
  15. Mathieu E, Ritchie H, Ortiz-Ospina E, Roser M, Hasell J, Appel C, et al. A global database of COVID-19 vaccinations. Nat Hum Behav 2021 Jul;5(7):947-953. [CrossRef] [Medline]
  16. Morahan-Martin JM. How internet users find, evaluate, and use online health information: a cross-cultural review. Cyberpsychol Behav 2004 Oct;7(5):497-510. [CrossRef] [Medline]
  17. Stockmeyer NO. Using Microsoft Word's readability program. Michigan Bar J 2009;88:46-47.
  18. D'Alessandro DM, Kingsley P, Johnson-West J. The readability of pediatric patient education materials on the World Wide Web. Arch Pediatr Adolesc Med 2001 Jul;155(7):807-812. [CrossRef] [Medline]
  19. Finn S. Unpredictability as correlate of reader enjoyment of news articles. Journalism Q 2016 Aug 29;62(2):334-345. [CrossRef]
  20. Prabhu AV, Donovan AL, Crihalmeanu T, Hansberry DR, Agarwal N, Beriwal S, et al. Radiology online patient education materials provided by major university hospitals: do they conform to NIH and AMA guidelines? Curr Probl Diagn Radiol 2018;47(2):75-79. [CrossRef] [Medline]
  21. Prabhu AV, Hansberry DR, Agarwal N, Clump DA, Heron DE. Radiation oncology and online patient education materials: deviating from NIH and AMA recommendations. Int J Radiat Oncol Biol Phys 2016 Nov 01;96(3):521-528. [CrossRef] [Medline]
  22. Boyer C, Baujard V, Geissbuhler A. Evolution of health web certification through the HONcode experience. Stud Health Technol Inform 2011;169:53-57. [Medline]
  23. Tools. Health On the Net.   URL: [accessed 2022-03-04]
  24. Schwarzbach HL, Mady LJ, Kaffenberger TM, Duvvuri U, Jabbour N. Quality and readability assessment of websites on human papillomavirus and oropharyngeal cancer. Laryngoscope 2021 Jan;131(1):87-94. [CrossRef] [Medline]
  25. Robillard JM, Jun JH, Lai J, Feng TL. The QUEST for quality online health information: validation of a short quantitative tool. BMC Med Inform Decis Mak 2018 Oct 19;18(1):87 [FREE Full text] [CrossRef] [Medline]
  26. Robillard JM, Feng TL. Health advice in a digital world: quality and content of online information about the prevention of Alzheimer's disease. J Alzheimers Dis 2017;55(1):219-229. [CrossRef] [Medline]
  27. Common questions about coronavirus vaccinations. National Health Commission of the People's Republic of China.   URL: [accessed 2021-07-08]
  28. FAQs on COVID-19 vaccine. Ministry of Health and Family Welfare, Government of India.   URL: [accessed 2021-07-08]
  29. Frequently asked questions about COVID-19 vaccination. Centers for Disease Control and Prevention.   URL: [accessed 2021-07-08]
  30. Perguntas e respostas. Governo do Brasil.   URL: [accessed 2021-07-08]
  31. Aktuelle informationen zur COVID-19-Impfung. Bundesministerium für Gesundheit.   URL: [accessed 2021-07-08]
  32. Coronavirus (COVID-19) vaccination. NHS.   URL: [accessed 2021-07-08]
  33. FAQ Covid-19. Santé.fr.   URL: [accessed 2021-07-08]
  34. Frequently asked questions. T.C. Sağlık Bakanlığı.   URL: [accessed 2021-07-08]
  35. FAQ - COVID-19 vaccines. Italian Medicines Agency.   URL: [accessed 2021-07-08]
  36. COVID-19 vaccines. Prime Minister of Japan and His Cabinet.   URL: [accessed 2021-07-08]
  37. Preguntas frecuentes. Vacuna Covid.   URL: [accessed 2021-07-08]
  38. Tanya jawab.   URL: [accessed 2021-07-08]
  39. Vacunación COVID-19 preguntas comunes. Ministerio de Sanidad.   URL: https:/​/www.​​en/​profesionales/​saludPublica/​prevPromocion/​vacunaciones/​covid19/​docs/​VACUNACION-COVID-19_PREGUNTAS.​pdf [accessed 2021-07-08]
  40. Ответы на самые распространенные вопросы о вакцинах и вакцинации. коронавирусе COVID-19: вакцина.   URL: https://xn--80aaezjt5d.xn--80aesfpebagmfblc0a.xn--p1ai/#faq [accessed 2021-07-08]
  41. Ask the experts COVID-19 vaccines questions: safety, ingredients and side effects.   URL: [accessed 2021-07-08]
  42. Pytania i odpowiedzi.   URL: [accessed 2021-07-08]
  43. Preguntas frecuentes.   URL: [accessed 2021-07-08]
  44. Preguntas frecuentes sobre la vacuna contra COVID-19.   URL: [accessed 2021-07-08]
  45. 코로나19 백신. 코로나19예방접종.   URL: [accessed 2021-07-08]
  46. Les réponses aux questions fréquentes posées autour de la vaccination contre la COVID-19. Campagne de vaccination contre le coronavirus au Maroc.   URL: [accessed 2021-07-08]
  47. Ministry of Health.   URL: [accessed 2021-07-08]
  48. COVID vaccination. National Command Operation Center.   URL: [accessed 2021-07-08]
  49. Getting vaccinated against COVID-19 in the Netherlands (translations). Government of the Netherlands.   URL: https:/​/www.​​topics/​coronavirus-covid-19/​documents/​leaflets/​2021/​01/​19/​getting-vaccinated-against-covid-19-in-the-netherlands [accessed 2021-07-08]
  50. About the vaccine. UAE Coronavirus (COVID-19) Updates.   URL: [accessed 2021-07-08]
  51. #ChecktheFAQs: a campaign to fight vaccine misinformation!. Department of Health, Republic of the Philippines.   URL: [accessed 2021-07-08]
  52. Frequently asked questions (FAQs). Coronavirus Covid-19.   URL: [accessed 2021-07-08]
  53. Thailand's Covid-19 vaccine. Department of Disease Control.   URL: [accessed 2021-07-08]
  54. COVID-19 questions and answers. [accessed 2021-07-08].
  55. Commonly asked. [accessed 2021-07-08].
  56. Popular questions. [accessed 2021-07-08].
  57. Vaccination | common questions. SNS24.   URL: [accessed 2021-07-08]
  58. Întrebări și răspunsuri. Guvernul Romaniei.   URL: [accessed 2021-07-08]
  59. Ερωτήσεις / Απαντήσεις για τον Εμβολιασμό του πληθυσμού απέναντι στον Covid-19. Αρχική Εμβολιασμός COVID-19.   URL: [accessed 2021-07-08]
  60. Vaccination against COVID-19. 1177 Vårdguiden.   URL: [accessed 2021-07-08]
  61. Questions and answers about vaccination. Ministry of Health of Czech Republic.   URL: [accessed 2021-07-08]
  62. Is it true? Get the facts on COVID-19 vaccines. Australian Government Department of Health.   URL: [accessed 2021-07-08]
  63. Mitos sobre las vacunas contra la COVID-19. Gobierno del Perú.   URL: [accessed 2021-07-08]
  64. COVID-19 vaccination. Bundesministerium für Soziales, Gesundheit, Pflege und Konsumentenschutz.   URL: [accessed 2021-07-08]
  65. Coronavirus: frequently asked questions (FAQs). Bundesamt für Gesundheit BAG.   URL: https:/​/www.​​bag/​en/​home/​krankheiten/​ausbrueche-epidemien-pandemien/​aktuelle-ausbrueche-epidemien/​novel-cov/​haeufig-gestellte-fragen.html?faq-url=/​covid/​en/​vaccination [accessed 2021-07-08]
  66. Что нужно знать о вакцине от COVID-19. Corona Virus 2020 KZ.   URL: [accessed 2021-07-08]
  67. FAQS - safety and efficacy of the COVID-19 vaccine. Ministry of Health Singapore.   URL: [accessed 2021-07-08]
  68. Vaccination against COVID-19. Sundhedsstyrelsen.   URL: https:/​/www.​​-/​media/​English/​Publications/​2021/​Corona/​Vaccination/​Vaccination_accessible-EN.​ashx?la=en&hash=48F7C36749F897CF57C0E0FFF9733EF042B49B5B [accessed 2021-07-08]
  69. Safety of COVID-19 vaccines. Terveyden ja hyvinvoinnin laitos.   URL: https:/​/thl.​fi/​en/​web/​infectious-diseases-and-vaccinations/​what-s-new/​coronavirus-covid-19-latest-updates/​vaccines-and-coronavirus/​safety-of-covid-19-vaccines [accessed 2021-07-08]
  70. COVID-19 vaccines: frequently asked questions. The Health Products Regulatory Agency.   URL: http:/​/www.​​homepage/​medicines/​covid-19-updates/​approval-of-covid-19-vaccines-frequently-asked-questions [accessed 2021-07-08]
  71. Coronavirus vaccine - information for the public. Folkehelseinstituttet.   URL: [accessed 2021-07-08]
  72. Preguntas frecuentes vacunación COVID-19. Sitio oficial de la República Oriental del Uruguay.   URL: https:/​/www.​​ministerio-salud-publica/​comunicacion/​publicaciones/​preguntas-frecuentes-vacunacion-covid-19/​preguntas-frecuentes-vacunacion [accessed 2021-07-08]
  73. سؤال وجواب.   URL: [accessed 2021-07-08]
  74. FAQs. The Government of the Hong Kong Special Administration Region.   URL: [accessed 2021-07-08]
  75. Coronavirus disease (COVID-19): vaccines safety. World Health Organization.   URL: [accessed 2021-07-08]
  76. COVID-19 vaccines: key facts. European Medicines Agency.   URL: https:/​/www.​​en/​human-regulatory/​overview/​public-health-threats/​coronavirus-disease-covid-19/​treatments-vaccines/​vaccines-covid-19/​covid-19-vaccines-key-facts [accessed 2021-07-08]
  77. Flesch R. How to Write Plain English: A Book for Lawyers and Consumers. New York, NY: Harper & Row; 1979.
  78. Fallis D, Frické M. Indicators of accuracy of consumer health information on the Internet: a study of indicators relating to information for managing fever in children in the home. J Am Med Inform Assoc 2002;9(1):73-79 [FREE Full text] [CrossRef] [Medline]
  79. Klugar M, Riad A, Mekhemar M, Conrad J, Buchbender M, Howaldt H, et al. Side effects of mRNA-based and viral vector-based COVID-19 vaccines among German healthcare workers. Biology (Basel) 2021 Aug 05;10(8):752 [FREE Full text] [CrossRef] [Medline]
  80. Riad A, Pokorná A, Attia S, Klugarová J, Koščík M, Klugar M. Prevalence of COVID-19 vaccine side effects among healthcare workers in the Czech Republic. J Clin Med 2021 Apr 01;10(7):1428 [FREE Full text] [CrossRef] [Medline]
  81. Zhang KC, Fang Y, Cao H, Chen H, Hu T, Chen Y, et al. Behavioral intention to receive a COVID-19 vaccination among Chinese factory workers: cross-sectional online survey. J Med Internet Res 2021 Mar 09;23(3):e24673 [FREE Full text] [CrossRef] [Medline]
  82. Wawrzuta D, Jaworski M, Gotlib J, Panczyk M. Characteristics of antivaccine messages on social media: systematic review. J Med Internet Res 2021 Jun 04;23(6):e24564 [FREE Full text] [CrossRef] [Medline]
  83. Dodd RH, Pickles K, Nickel B, Cvejic E, Ayre J, Batcup C, et al. Concerns and motivations about COVID-19 vaccination. Lancet Infect Dis 2021 Feb;21(2):161-163 [FREE Full text] [CrossRef] [Medline]
  84. Cuan-Baltazar JY, Muñoz-Perez MJ, Robledo-Vega C, Pérez-Zepeda MF, Soto-Vega E. Misinformation of COVID-19 on the internet: infodemiology study. JMIR Public Health Surveill 2020 Apr 09;6(2):e18444 [FREE Full text] [CrossRef] [Medline]
  85. Zhang M, Chow A, Smith H. COVID-19 contact-tracing apps: analysis of the readability of privacy policies. J Med Internet Res 2020 Dec 03;22(12):e21572 [FREE Full text] [CrossRef] [Medline]

FAQ: frequently asked question
HONcode: Health On the Net Foundation code
QUEST: Quality Evaluation Scoring Tool
Q&A: question and answer
WHO: World Health Organization

Edited by T Sanchez; submitted 03.10.21; peer-reviewed by S Bucher, M Das; comments to author 17.10.21; revised version received 27.10.21; accepted 20.01.22; published 15.03.22


©Andy Wai Kan Yeung, Thomas Wochele-Thoma, Fabian Eibensteiner, Elisabeth Klager, Mojca Hribersek, Emil D Parvanov, Dalibor Hrg, Sabine Völkl-Kernstock, Maria Kletecka-Pulker, Eva Schaden, Harald Willschke, Atanas G Atanasov. Originally published in JMIR Public Health and Surveillance (, 15.03.2022.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.