Published on in Vol 7, No 12 (2021): December

Preprints (earlier versions) of this paper are available at, first published .
Learning From a Massive Open Online COVID-19 Vaccination Training Experience: Survey Study

Learning From a Massive Open Online COVID-19 Vaccination Training Experience: Survey Study

Learning From a Massive Open Online COVID-19 Vaccination Training Experience: Survey Study

Original Paper

1Influenza Preparedness and Response, Organisation Mondiale de la Santé, Genève, Switzerland

2Strategic Research, Laerdal Medical, Stavanger, Norway

3Learning and Capacity Development Unit, WHO Health Emergencies Program, Organisation Mondiale de la Santé, Genève, Switzerland

4Department of Immunization, Vaccines and Biologicals, Organisation Mondiale de la Santé, Genève, Switzerland

Corresponding Author:

Shoshanna Goldin, MPH

Influenza Preparedness and Response

Organisation Mondiale de la Santé

20 Avenue Appia

Genève, CH-1211


Phone: 41 795165651


Background: To prepare key stakeholders for the global COVID-19 vaccination rollout, the World Health Organization and partners developed online vaccination training packages. The online course was launched in December 2020 on the OpenWHO learning platform. This paper presents the findings of an evaluation of this course.

Objective: The aim of this evaluation was to provide insights into user experiences and challenges, measure the impact of the course in terms of knowledge gained, and anticipate potential interest in future online vaccination courses.

Methods: The primary source of data was the anonymized information on course participants, enrollment, completion, and scores from the OpenWHO platform’s statistical data and metric reporting system. Data from the OpenWHO platform were analyzed from the opening of the courses in mid-December 2020 to mid-April 2021. In addition, a learner feedback survey was sent by email to all course participants to complete within a 3-week period (March 19 to April 9, 2021). The survey was designed to determine the perceived strengths and weaknesses of the training packages and to understand barriers to access.

Results: During the study period, 53,593 learners enrolled in the course. Of them, 30,034 (56.0%) completed the course, which is substantially higher than the industry benchmark of 5%-10% for a massive open online course (MOOC). Overall, learners averaged 76.5% on the prequiz compared to 85% on the postquiz, resulting in an increase in average score of 9%. A total of 2019 learners from the course participated in the survey. Nearly 98% (n=1647 fully agree, n=308 somewhat agree; N=1986 survey respondents excluding missing values) of respondents fully or somewhat agreed that they had more confidence in their ability to support COVID-19 vaccination following completion of this course.

Conclusions: The online vaccine training was well received by the target audience, with a measurable impact on knowledge gained. The key benefits of online training were the convenience, self-paced nature, access to downloadable material, and ability to replay material, as well as an increased ability to concentrate. Online training was identified as a timely, cost-effective way of delivering essential training to a large number of people to prepare for the COVID-19 vaccination rollout.

JMIR Public Health Surveill 2021;7(12):e33455



To address the need for timely training on COVID-19 vaccination, the Access to COVID-19 Tools Accelerator’s Country Readiness and Delivery (CRD) workstream rapidly produced the COVID-19 vaccination training for health workers course. The course was launched on the OpenWHO platform in mid-December 2020. The objective was to ensure that health workers responsible for COVID-19 vaccination deployment had timely access to World Health Organization (WHO) recommendations and information that could help them prepare for a safe and efficient vaccine rollout. Although the course was targeted to health workers, it was open to all and was accessed by others, such as policy makers, community leaders, and students.

As the COVID-19 pandemic limited travel and the ability to gather learners together in a typical classroom environment, the health workers course was developed as an online learning curriculum. The health workers course includes a series of video lectures presented by technical experts, with accompanying multiple-choice questions delivered before and after the lectures. The transcripts and videos are downloadable. The health workers course contains 6 modules and provides information on organizing the vaccination session, including infection prevention and control measures; COVID-19 vaccine storage, handling, administration, and safe disposal; recording and monitoring, including adverse events following immunization (AEFI); and communication with the community.

OpenWHO is one of the WHO’s online learning platforms, offering free online courses with the aim to improve responses to health emergencies [1]. The platform hosts over 100 courses on COVID-19 and other health topics, has over 5 million enrollments, and offers courses in 55 languages. Within the OpenWHO platform, registration includes an option to self-declare the registrant's occupation. The occupation selection is not validated and may be subjective, depending on the registrant’s consideration of their role in the workforce.

Given that online learning to prepare for vaccine introduction is a relatively new approach for many countries, this course was evaluated to understand participants’ online learning experience. The aim of this analysis is to increase the effectiveness of OpenWHO training packages and to plan for future online immunization learning.

The COVID-19 pandemic emphasized the need for trusted, accurate information to help health workers and the public respond to the outbreak. Online learning to prepare for vaccine introduction is a relatively new approach [2]. This paper provides an overview of the OpenWHO COVID-19 vaccine introduction training for health workers, shares insights on participants’ online learning experiences, and provides key findings that can be used to improve future real-time online training.

The title of the course (COVID-19 vaccination training for health workers) uses the WHO definition of health workers as all people engaged in actions whose primary intent is to enhance health.

The analysis is based on quantitative data collected from the OpenWHO integrated statistical data and analytics reporting system. Anonymized course reports data sets were extracted from the OpenWHO reporting tool, providing raw data including basic demographics on OpenWHO users (eg, self-declared age, gender, professional affiliation, and nationality information), which were entered at the time of registration to the platform. OpenWHO course reports also include course-specific learners’ performance and course activity indicators (such as module completion), including videos, self-assessments, download activity status, quiz performance, and obtention of the certificate, as well as tracked average session duration and time-stamped activity usage patterns.

Course registration and completion data captured by the OpenWHO analytics systems were analyzed to understand user demographics, certifications, and dropout rates. Completion of the course was defined as watching all videos and completing the postquiz with a score of at least 80%. Questions were scored as correct or incorrect—no partial credit was granted. Demographic information, including age, gender, and professional affiliation were not mandatory; therefore, analyses on these variables were based on the learners who provided information voluntarily. The course activity data of individual learners—including modules visited, videos watched, and resources downloaded—were collected and analyzed to understand the usability of the training course. Data on the scores of individual learners for pre- and postquizzes were collected and used in the analysis to measure knowledge gained. Learner’s knowledge gain was assessed by comparing average postquiz scores to prequiz scores (where data were available). Both pre- and postquizzes had the same questions. Pre- and postquizzes were included before and after each module, respectively. The number of questions was limited (2-4 questions per module) to avoid overburdening the learners. Learners had a single attempt for prequizzes, but multiple attempts were allowed for postquizzes. For each learner’s postquiz scores, the scores from the first postquiz attempts were used. Due to the limited number of questions per module, statistical significance by module could not be analyzed. Analysis of the course data was based on the total number of enrolled learners from the course opening date (December 18, 2020) until the date when the course analyses for this paper began (April 18, 2021).

In addition, an exit survey was added at the end of the course to collect participant feedback on course content to better understand the usability and virtual learning experiences of the learners, as well as strengths, weaknesses, and barriers of the training package. The learner feedback survey was composed of 21 questions (Multimedia Appendix 1). The survey was implemented on the OpenWHO platform and opened for a period of 3 weeks (March 19-April 9, 2021). For those learners who enrolled and completed the health workers course prior to March 19, 2021, survey invitation emails were sent. The survey was voluntary and indicated that it was conducted to collect feedback and that results may be used for research purposes.

The two sets of anonymized statistical data from the course report and survey data were overlaid by using a unique pseudo-ID for each OpenWHO learner, thus allowing the two data sets to be merged for analysis.

All analyses were conducted using Python (version 3.8.3; Python Software Foundation). OpenWHO data and the survey data were collected in line with the OpenWHO Terms of Use, which every enrolled user accepts. All OpenWHO users agree to the following statement, which was provided by the Office of the Legal Counsel of the WHO: “Records of your participation in OpenWHO courses may be used for education research. In the interest of this research, you may be exposed to variations in the course content. Research findings will typically be reported at the aggregate level. Your personal identity will not be publicly disclosed in any research findings without your express consent.” As the survey was conducted to provide feedback on the course, ethical clearance was not required.

Summary Statistics

During the study period, the total number of enrolled learners was 53,595. Of all enrolled learners in the health workers course, 30,034 (56.0%) completed the course. Out of 2019 survey participants, 1857 (92.0%) completed the course and 432 (8.0%) did not. All survey responses were included in the survey analysis, as this paper considers the knowledge gained and experiences of participants who did and did not complete the course.

Demographic characteristics of the enrolled learners for the course are described in Table 1. Of the enrolled learners, 34,746 (64.8%) provided their gender, 33,557 (62.6%) provided their age, and 46,909 (87.5%) provided their professional affiliation. There were more female learners (n=18,388, 52.9%) than male learners (n=16,311, 47.0%) and the age group of 20-29 years was most dominant in the course (n=11,444, 34.1%). The top 3 professional affiliations of learners were the following: health care professionals (n=21,487, 45.8%), students (n=6874, 14.7%), and ministry of health officials (n=4666, 9.9%).

The course was also translated into 11 additional languages: Arabic, Chinese, Dutch, French, Indonesian, Kazak, Macedonian, Spanish, Russian, Portuguese, and Vietnamese. This analysis considers the English version, as it was the original course launched and it has the largest enrollment of the language versions. Learners from 191 countries participated in the English version of the course. The four countries with the highest number of participants in the English course were India (n=3998, 11.8%), Philippines (n=2700, 7.9%), Nigeria (n=2297, 6.8%), and Rwanda (n=2163, 6.4%).

When asked about their motivation to enroll in the health workers course, 58.5% (n=1177) of survey respondents replied that they participated in the course to prepare themselves for specific professional responsibilities, 12.6% (n=255) out of private interest, 10.9% (n=220) because it was mandatory, 8.0% (n=161) to strengthen their resume, 7.0% (n=142) to be able to teach others, and 3.0% (n=64) for other and unknown reasons. However, there were substantial differences in motivation between countries, professional affiliations, and years of experience (Figure 1 and Table S1 in Multimedia Appendix 2 for numerical values). For example, most learners from the United States were students and took the course because it was required. In comparison, a large proportion of the learners from Ministries of Health took the course to be able to teach others.

Table 1. Demographic characteristics of the total enrolled learners in the English version of the COVID-19 vaccination training for health workers course on OpenWHO from December 18, 2020, to April 18, 2021.
CharacteristicsValues, n (%)
Total enrollments53,595 (100)
Number of learners that completed the course30,034 (56)

Female18,388 (52.9)

Male16,311 (47)

Other47 (0.1)
Age group (years)a

<201279 (3.8)

20-2911,444 (34.1)

30-3910,756 (32)

40-495711 (17)

50-593146 (9.4)

60-691094 (3.3)

>70127 (0.4)
Professional affiliation (top 3)a

Health care professionals21,487 (45.8)

Students6874 (14.7)

Health ministry4666 (9.9)
Country of residence (top 4)a

India3998 (11.8)

Philippines2700 (7.9)

Nigeria2297 (6.8)

Rwanda2163 (6.4)

aAmong those who provided information.

Figure 1. Survey participants' motivation for taking the health workers course by selected country, professional affiliation, and years of experience (total survey participants=2019; survey period from March 19 to April 9, 2021). MoH: Ministry of Health; USA: United States of America.
View this figure

Usability of Course

Survey respondents indicated that they primarily watched the videos as the main resource for the training, which corresponds to the intended training method. Based on the total enrollment course data, on average, Module 2, which focused on supply chains and logistics, was the most watched video of the course, while Module 5, which focused on reporting and monitoring COVID-19 vaccination, was the least watched video among all enrolled learners (Figure 2). More than half of the survey respondents also read the transcripts (n=1070, 53.0%) and downloaded the presentations (n=1083, 53.6%).

For all modules, the postquiz was the course component deemed most useful by the enrolled learners. Although the quizzes were not mandatory, the postquizzes were highly used.

Males aged 40-49 years and females aged 50- 59 years were most likely to complete the course. The demographics of learners least likely to complete the course were females under the age of 20 years and learners from a health expert group or other ministry.

For this course, learners spent a median of 25.7 minutes per session and typically completed the course in three sessions (total duration of 72.1 minutes). As the run time for all videos in this course is approximately 1.5 hours, learners may have played videos at a faster speed (OpenWHO allows for participants to speed up the videos by up to 2 times the speed of the original recording) or skipped some parts of the videos.

Figure 2. Usability (by average percent completion) of modules by total enrolled learners (n=53,595) in the COVID-19 vaccination training for health workers course on OpenWHO from December 18, 2020, to April 18, 2021. AEFI: adverse events following immunization.
View this figure

Knowledge and Confidence Gained

As the increase in scores varied by module, the breakdown by module is included below, along with the average change in score from pre- to postquiz. Notably, the module with the highest increase in score was Module 4, which focuses on AEFI monitoring. In Module 4, learners averaged 62% on the prequiz and 78% on the postquiz, for an overall increase of 16%. Overall, learners averaged 76% on the prequiz compared to 85% on the postquiz, resulting in an increase in average score of 9% (Table 2).

Survey respondents were asked whether they had more confidence in their ability to perform their professional role related to COVID-19 vaccination after this training. Overall, nearly 83% (n=1647) of respondents fully agreed with this statement and an additional 15.5% (n=308) somewhat agreed with this statement. Among the top 3 professional affiliation groups, health care professionals were most likely to fully agree with this statement (n=1136, 85.4%) compared to students (n=171, 77.7%) and those working in the public health sector (n=66, 71.0%). Individuals with 4-6 years of experience in their field had the highest “fully agree” rate (n=240, 86.1%), while those with <1 year of experience had the lowest “fully agree” rate (n=196, 78.7%).

Table 2. Pre- and postquiz scores of 53,595 enrolled learners for the COVID-19 vaccination training for health workers course on OpenWHO from December 18, 2020, to April 18, 2021.
ModulesPrequiz scorea, %Postquiz scorea, %Improvement, %Number of questions
Module 1: Introduction to COVID-19 vaccination training889352
Module 2: Storage, handling, delivery, and waste management of COVID-19 vaccines818872
Module 3: Organizing COVID-19 vaccination sessions7285134
Module 4: AEFIb monitoring for COVID-19 vaccination6278163
Module 5: Recording and monitoring COVID-19 vaccination758492
Module 6: Communication with the community about COVID-19 vaccination818102

aAmong those who participated in the quiz.

bAEFI: adverse events following immunization.

Considerations for Future Virtual Trainings

The health workers course was particularly well received by health care professionals. About 99% (n=1966) of survey respondents indicated that they would recommend this course to others, with 91.3% (n=1832) fully agreeing and 7.6% (n=134) somewhat agreeing. Among the top 3 professional affiliation groups, health care professionals had the highest fully agree rate (n=1244, 93.7%), while students had the lowest (n=188, 86.2%).

When asked their preferred training methods, 65.6% (n=1293) of the survey respondents preferred online training, 29.3% (n=579) preferred blended (combination of online and in-person training), 3.7% (n=73) preferred in-person training, and 1.4% (n=27) were unsure. The reasons respondents preferred online training (multiple responses were possible) included the convenience of the timing (n=1204, 59.6%), the self-paced nature (n=1039, 51.5%), the ability to download the materials (n=907, 44.9%), the ability to replay sections (n=890, 44.1%), and the increased ability to concentrate (n=520, 25.8%). As these responses were from learners who participated in the online training and completed the online survey, it is important to note that this may reflect an overestimate of overall willingness to participate in online learning among the general population.

When asked about areas for future improvement, 32.9% (n=604) of survey respondents requested that OpenWHO offer more COVID-19 vaccination courses (particularly vaccine-specific resources).

In addition, 17.9% (n=362) of survey respondents requested the course be available in their national language and 4% (n=81) asked for more interaction with technical experts.

At the time of the submission of this article (July 2021), the health workers course has been provided in 12 languages and has had more than 110,000 enrollments (Table 3). The average completion rate is 65% and the highest is 89%, for the Spanish-language course.

Table 3. Total number of enrollments and completion certificates awarded by language version for the COVID-19 vaccination training for health workers course on OpenWHO from December 18, 2020, to April 18, 2021.
Course number in order of launch dateLanguageNumber of enrollmentsNumber of certificates (completion rate)

1English68,26737,284 (55)
2Bahasa46602786 (60)
3Russian350159 (45)
4Macedonian8439 (46)
5Chinese424240 (57)
6Arabic1324665 (50)
7Spanish32,67229,178 (89)
8French1959970 (50)
9Portuguese501209 (42)
10Vietnamese394195 (49)
11Dutch18244 (24)
12Kazakh191 (5)

Principal Findings

Although the completion rate for this course was substantially higher than the industry benchmark of 5%-10% for a massive open online course (MOOC), the findings from this OpenWHO course correspond to other online training experiences for OpenWHO and other virtual training platforms [3-9].

Health care professionals are OpenWHO’s largest user group, accounting for nearly one-third of users [10]. In the context of COVID-19 vaccination, primary health workers may serve as “knowledge ambassadors” [11] or “knowledge brokers” [12] and, as such, may have the greatest chance to increase confidence about the vaccine among their patients. For example, it was demonstrated that the acceptability of the COVID-19 vaccine was greater among individuals who thought their health care provider would recommend it [13].

Online and blended learning can provide substantial cost savings by reducing the need for travel, per diems, and other related expenses, as well as rapidly increasing the potential number of people that can be trained [14,15]. As online learning is still relatively new for the training of health workers, the modality has received mixed reviews. Several systematic reviews report that online learning approaches may be at least as effective as traditional learning approaches [11-14], while others show that online learning may make little or no difference in patient outcomes or health professionals’ behaviors, skills, or knowledge [16]. However, included studies have used different study designs to measure the effectiveness of online learning, from cross-sectional approaches with pre- and post-test assessments (ie, testing before and after the learning activity) [17] to longitudinal research, where knowledge retention was assessed up to 6 months or a 1-year follow up was carried out [18], which makes comparing these studies difficult.

Although the health workers course was well received, feedback did include the need for additional vaccine-specific training content, more translated versions of the course, opportunities to ask questions to technical experts, and the ability to participate in peer-to-peer learning. Following the request for more vaccine-specific content, CRD launched the vaccine-specific resources course in all United Nations languages (Arabic, Chinese, English, French, Russian, and Spanish) as well as Portuguese [19]. This course provides short instructional videos for COVID-19 vaccines that received Emergency Use Listing, such as Pfizer-BioNTech, Moderna, AstraZeneca, and Janssen. In addition, this new course provides job aides (ie, resources providing vaccine-specific information) to support stakeholders involved in COVID-19 vaccine deployment. To support the development of additional language versions, CRD worked with WHO Country Offices to provide translated versions of both this OpenWHO course for health workers and the Orientation to National Deployment and Vaccination Planning for COVID-19 vaccines course [20]. In response to participants’ requests for more interaction and peer learning, CRD developed and implemented the COVID-19 Vaccination: Building Global Capacity webinar series, which brought together technical experts and learners for 15 live sessions dedicated to different aspects of COVID-19 vaccination. This webinar series ultimately reached more than 13,000 learners in 181 countries.

Ideally, virtual training could include recorded and live components, allowing for a combination of the flexibility offered by virtual self-paced learning with the opportunity to interact during the live sessions [21-25]. Considering the speed with which learners completed this course, it may be beneficial to provide shorter versions of the content.

In addition, when considering virtual courses, internet connectivity and the potential for system- or IT-related issues are important to consider, in particular at the subnational level in low-resource settings. In our analysis, nearly one-third of survey respondents noted that they had at least some internet connection issues during their learning. Similarly, recent research demonstrated that accessibility of online learning activities may be hampered by the required baseline level of digital literacy, equipment, and internet connection, which might be of particular importance for certain populations, including refugees or people with vision problems and those living in low-resource settings [26-29]. If the WHO, governments, and partners plan to increase the use of online learning, it is critical to also consider the infrastructure necessary to ensure learners can fully participate.

Limitations of this research include that we focused this analysis on the English-speaking course. Additional analyses could be conducted on the other language versions of the course. This research also reflects a snapshot in time, as the survey was conducted in March and April 2021. Follow-up surveys could be conducted a year or two after the launch of the course to understand the evolution of the course experience and to understand how participants used the information they received from the course in their professional and personal lives. An additional limitation of these findings is the potential bias of people more comfortable with online learning having taken the course and completed the survey. Finally, this analysis may include potential self-reporting bias among survey respondents, while the limited number of questions in the pre- and postquizzes precludes robust statistical analysis of the impact of each module. It would be beneficial for future analyses to consider differences in the characteristics and perspectives of participants who complete virtual courses compared with participants who do not complete virtual courses.

Overall, this analysis highlights a strong interest in online learning among participating health professionals. This willingness to participate in virtual training is important for the WHO and partners to consider when developing educational materials for other vaccine introductions. Online learning may serve as a viable alternative to face-to-face training, particularly in an emergency context when physical distancing is recommended. It would be beneficial for future studies to look at how health workers applied the knowledge gained from this training and to consider the cost-effectiveness and/or cost-benefit of online learning for vaccine introduction, particularly during health emergencies.


The COVID-19 vaccination trainings were developed for OpenWHO due to the global need for rapidly available training, the need for rapid dissemination to a large number of learners, and the travel and operational limitations posed by the pandemic. This article provides an overview of the usability and utility of this global virtual training, as well as insights from the experience.

In summary, this analysis indicates that this course served its intended purpose of supporting participating health workers in preparing for COVID-19 vaccination deployment. Considering this analysis and the increasing desire of learners to have training materials and performance scores rapidly accessible, Ministries of Health and health facilities should consider the potential of training their health professionals using virtual or blended approaches to increase rapid accessibility and exchange of information [23,25].


The World Health Emergencies OpenWHO team provided the funding for open-source publication of this article.

We would like to acknowledge the CRD’s Communications, Advocacy, and Training working group for their insight, the CRD leadership for their support, Dr Aleksandra Caric and Ms Richelle George for their assistance in developing these training materials, and Gavi, the Vaccine Alliance, which enabled Laerdal Medical to provide data analysis support for the survey and this article.

Conflicts of Interest

All authors were surveyed for potential conflicts of interest using the WHO Declaration of Interest form. No conflicts of interest were identified. Laerdal Medical (SYJK) was contracted by the WHO to conduct this analysis.

Multimedia Appendix 1

Survey questions.

PDF File (Adobe PDF File), 213 KB

Multimedia Appendix 2

Supplementary table.

PDF File (Adobe PDF File), 136 KB

  1. Bhattacharya S, Singh A, Hossain M. Health system strengthening through Massive Open Online Courses (MOOCs) during the COVID-19 pandemic: An analysis from the available evidence. J Educ Health Promot 2020;9(1):195 [FREE Full text] [CrossRef] [Medline]
  2. Serino L, Maurici M, D'Alò GL, Amadori F, Terracciano E, Zaratti L, Local Health Units Vaccination Group. Healthcare Workers Training Courses on Vaccinations: A Flexible Format Easily Adaptable to Different Healthcare Settings. Vaccines (Basel) 2020 Sep 08;8(3):514 [FREE Full text] [CrossRef] [Medline]
  3. Murugesan R, Nobes A, Wild J. A MOOC approach for training researchers in developing countries. Open Praxis 2017 Mar 31;9(1):45-57 [FREE Full text]
  4. Liyanagunawardena TR, Adams AA, Williams SA. MOOCs: A systematic study of the published literature 2008-2012. IRRODL 2013 Jul 05;14(3):202. [CrossRef]
  5. Jordan K. Initial trends in enrolment and completion of massive open online courses. IRRODL 2014 Jan 15;15(1):1651. [CrossRef]
  6. Hew KF, Cheung WS. Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges. Educational Research Review 2014 Jun;12:45-58. [CrossRef]
  7. Sneddon J, Barlow G, Bradley S, Brink A, Chandy S, Nathwani D. Development and impact of a massive open online course (MOOC) for antimicrobial stewardship. J Antimicrob Chemother 2018 Apr 01;73(4):1091-1097. [CrossRef] [Medline]
  8. Veletsianos G, Shepherdson P. A Systematic Analysis and Synthesis of the Empirical MOOC Literature Published in 2013–2015. IRRODL 2016 Mar 01;17(2):2448. [CrossRef]
  9. Li L, Lin M, Wang X, Bao P, Li Y. Preparing and responding to 2019 novel coronavirus with simulation and technology-enhanced learning for healthcare professionals: challenges and opportunities in China. BMJ Simul Technol Enhanc Learn 2020 Jul;6(4):196-198 [FREE Full text] [CrossRef] [Medline]
  10. Utunen H, George R, Ndiaye N, Tokar A, Attias M, Gamhewage G. Delivering WHO's Life-Saving Information in Real-Time During a Pandemic Through an Online Learning Platform: Evidence from Global Use. Stud Health Technol Inform 2021 May 27;281:969-973. [CrossRef] [Medline]
  11. Brown A, Barnes C, Byaruhanga J, McLaughlin M, Hodder RK, Booth D, et al. Effectiveness of Technology-Enabled Knowledge Translation Strategies in Improving the Use of Research in Public Health: Systematic Review. J Med Internet Res 2020 Jul 31;22(7):e17274 [FREE Full text] [CrossRef] [Medline]
  12. Martinengo L, Yeo NJY, Markandran KD, Olsson M, Kyaw BM, Car LT. Digital health professions education on chronic wound management: A systematic review. Int J Nurs Stud 2020 Apr;104:103512. [CrossRef] [Medline]
  13. Tudor Car L, Soong A, Kyaw BM, Chua KL, Low-Beer N, Majeed A. Health professions digital education on clinical practice guidelines: a systematic review by Digital Health Education collaboration. BMC Med 2019 Jul 18;17(1):139 [FREE Full text] [CrossRef] [Medline]
  14. Christensen L, Rasmussen CS, Benfield T, Franc JM. A Randomized Trial of Instructor-Led Training Versus Video Lesson in Training Health Care Providers in Proper Donning and Doffing of Personal Protective Equipment. Disaster Med Public Health Prep 2020 Aug 30;14(4):514-520 [FREE Full text] [CrossRef] [Medline]
  15. Razai MS, Kankam HKN, Hourston GJM, Hoppe S, Oakeshott P. Lessons Learned From Producing Guidance Articles and Rapid Massive Open Online Courses During the Covid-19 Pandemic in Primary Care. J Prim Care Community Health 2020 Oct 01;11:2150132720963624 [FREE Full text] [CrossRef] [Medline]
  16. Smith IM, Bayliss E, Salisbury H, Wheeler A. Operations management on the front line of COVID-19 vaccination: building capability at scale via technology-enhanced learning. BMJ Open Qual 2021 Jul 08;10(3):e001372 [FREE Full text] [CrossRef] [Medline]
  17. Vaona A, Banzi R, Kwag HK, Rigon G, Cereda D, Pecoraro V, et al. E-learning for health professionals. Cochrane Database Syst Rev 2018 Jan 21;1:CD011736 [FREE Full text] [CrossRef] [Medline]
  18. Maier A, Hoffman U, Fegert JM. Child protection in medicine - closing gaps in continuing education through e-learning. The European Journal of Public Health 2019;29(4):ckz186.567. [CrossRef]
  19. OpenWHO: COVID-19 specific-vaccine resources.   URL: [accessed 2021-10-01]
  20. OpenWHO: Orientation to national deployment and vaccination planning for COVID-19 vaccines.   URL: [accessed 2021-11-01]
  21. Colaceci S, Zambri F, D'Amore C, De Angelis A, Rasi F, Pucciarelli G, et al. Long-Term Effectiveness of an E-Learning Program in Improving Health Care Professionals' Attitudes and Practices on Breastfeeding: A 1-Year Follow-Up Study. Breastfeed Med 2020 Apr 01;15(4):254-260. [CrossRef] [Medline]
  22. Ravin CR. Implementation of a journal club on adult learning and nursing professional development. J Contin Educ Nurs 2012 Oct;43(10):451-455. [CrossRef] [Medline]
  23. Johnson SD, Aragon SR. An instructional strategy framework for online learning environments. New Directions for Adult and Continuing Education 2003 Nov 20;2003(100):31-43. [CrossRef]
  24. Regmi K, Jones L. A systematic review of the factors - enablers and barriers - affecting e-learning in health sciences education. BMC Med Educ 2020 Mar 30;20(1):91 [FREE Full text] [CrossRef] [Medline]
  25. Zitzmann NU, Matthisson L, Ohla H, Joda T. Digital Undergraduate Education in Dentistry: A Systematic Review. Int J Environ Res Public Health 2020 May 07;17(9):3269 [FREE Full text] [CrossRef] [Medline]
  26. Arghode V, Brieger EW, McLean GN. Adult learning theories: implications for online instruction. EJTD 2017 Aug 07;41(7):593-609. [CrossRef]
  27. Launois P, Maher D, Certain E, Ross BJM, Penkunas MJ. Implementation research training for learners in low- and middle-income countries: evaluating behaviour change after participating in a massive open online course. Health Res Policy Syst 2021 Apr 06;19(1):59 [FREE Full text] [CrossRef] [Medline]
  28. Burkardt AD, Krause N, Rivas Velarde MC. Critical success factors for the implementation and adoption of e-learning for junior health care workers in Dadaab refugee camp Kenya. Hum Resour Health 2019 Dec 09;17(1):98 [FREE Full text] [CrossRef] [Medline]
  29. Fernández-Díaz E, Iglesias-Sánchez PP, Jambrino-Maldonado C. Exploring WHO Communication during the COVID 19 Pandemic through the WHO Website Based on W3C Guidelines: Accessible for All? Int J Environ Res Public Health 2020 Aug 05;17(16):5663 [FREE Full text] [CrossRef] [Medline]

AEFI: adverse events following immunization
CRD: Country Readiness and Delivery
MOOC: massive open online course
WHO: World Health Organization

Edited by T Sanchez; submitted 11.09.21; peer-reviewed by T Prakash, K Medica, N Zeeni; comments to author 28.09.21; revised version received 18.10.21; accepted 16.11.21; published 03.12.21


©Shoshanna Goldin, So Yeon Joyce Kong, Anna Tokar, Heini Utunen, Ngouille Ndiaye, Jhilmil Bahl, Ranil Appuhamy, Ann Moen. Originally published in JMIR Public Health and Surveillance (, 03.12.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.