This paper is in the following e-collection/theme issue:

Mass Media/Social Media Communication and Campaigns (446) JMIR Theme Issue: COVID-19 Special Issue (2388) Outbreak and Pandemic Preparedness and Management (1673) Infoveillance, Infodemiology, Digital Disease Surveillance, Infodemic Management (1010) Infectious Diseases (non-STD/STI) (1478) Medicine 2.0: Social Media, Open, Participatory, Collaborative Medicine (1552) Infodemiology and Infoveillance (1269) Theme Issue: Novel Coronavirus (COVID-19) Outbreak Rapid Reports (1481)

Published on in Vol 7, No 7 (2021): July

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/27942, first published .
The Saudi Ministry of Health’s Twitter Communication Strategies and Public Engagement During the COVID-19 Pandemic: Content Analysis Study

The Saudi Ministry of Health’s Twitter Communication Strategies and Public Engagement During the COVID-19 Pandemic: Content Analysis Study

The Saudi Ministry of Health’s Twitter Communication Strategies and Public Engagement During the COVID-19 Pandemic: Content Analysis Study

Authors of this article:

Fatimah Mohammed Alhassan1, 2 Author Orcid Image ;   Sharifah Abdullah AlDossary1, 2 Author Orcid Image
Article Authors Cited by (23) Tweetations (16) Metrics

    Original Paper

    1Department of Health Informatics, College of Public Health and Health Informatics, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia

    2King Abdullah International Medical Research Center, Riyadh, Saudi Arabia

    Corresponding Author:

    Fatimah Mohammed Alhassan, BSc, MSc

    Department of Health Informatics

    College of Public Health and Health Informatics

    King Saud bin Abdulaziz University for Health Sciences

    PO Box 3660

    Riyadh, 11481

    Saudi Arabia

    Phone: 966 11 4299999

    Email: fatimah.m.alhasan@gmail.com


    Background: During a public health crisis such as the current COVID-19 pandemic, governments and health authorities need quick and accurate methods of communicating with the public. While social media can serve as a useful tool for effective communication during disease outbreaks, few studies have elucidated how these platforms are used by the Ministry of Health (MOH) during disease outbreaks in Saudi Arabia.

    Objective: Guided by the Crisis and Emergency Risk Communication model, this study aimed to explore the MOH’s use of Twitter and the public’s engagement during different stages of the COVID-19 pandemic in Saudi Arabia.

    Methods: Tweets and corresponding likes and retweets were extracted from the official Twitter account of the MOH in Saudi Arabia for the period of January 1 through August 31, 2020. Tweets related to COVID-19 were identified; subsequently, content analysis was performed, in which tweets were coded for the following message types: risk messages, warnings, preparations, uncertainty reduction, efficacy, reassurance, and digital health responses. Public engagement was measured by examining the numbers of likes and retweets. The association between outbreak stages and types of messages was assessed, as well as the effect of these messages on public engagement.

    Results: The MOH posted a total of 1393 original tweets during the study period. Of the total tweets, 1293 (92.82%) were related to COVID-19, and 1217 were ultimately included in the analysis. The MOH posted the majority of its tweets (65.89%) during the initial stage of the outbreak. Accordingly, the public showed the highest level of engagement (as indicated by numbers of likes and retweets) during the initial stage. The types of messages sent by the MOH significantly differed across outbreak stages, with messages related to uncertainty reduction, reassurance, and efficacy being prevalent among all stages. Tweet content, media type, and crisis stage influenced the level of public engagement. Engagement was negatively associated with the inclusion of hyperlinks and multimedia files, while higher level of public engagement was associated with the use of hashtags. Tweets related to warnings, uncertainty reduction, and reassurance received high levels of public engagement.

    Conclusions: This study provides insights into the Saudi MOH’s communication strategy during the COVID-19 pandemic. Our results have implications for researchers, governments, health organizations, and practitioners with regard to their communication practices during outbreaks. To increase public engagement, governments and health authorities should consider the public’s need for information. This, in turn, could raise public awareness regarding disease outbreaks.

    JMIR Public Health Surveill 2021;7(7):e27942

    doi:10.2196/27942

    Keywords

    COVID-19; Crisis and Emergency Risk Communication; effective communication; health authorities; outbreak; pandemic; public engagement; public health; social media; Twitter 


    Background

    Coronaviruses are a large family of viruses that cause diseases ranging from those with common cold symptoms to more severe pneumonia-like illnesses [1]. On December 31, 2019, the World Health Organization (WHO) Country Office in China declared that a new coronavirus, SARS-CoV-2, had been detected in Wuhan. Within a few weeks, the virus had spread from Wuhan to many provinces within China. It subsequently spread outside China, reaching over 200 countries. The rapid and continuous spread of the virus led the WHO to declare COVID-19, caused by SARS-CoV-2, a public health emergency of international concern on January 30, 2020, and a pandemic on March 11, 2020 [2-5].

    Saudi Arabia is the second largest Arab country with a population of over 34 million people [6]. The Ministry of Health (MOH) in Saudi Arabia is the largest provider of health care services, providing approximately 60% of the health care services nationwide, while the remainder is covered by other governmental and private facilities [7]. Since the confirmation of the first case of COVID-19 in Saudi Arabia on March 2, 2020, the government has taken prompt and decisive measures to combat the outbreak. These measures included, but were not limited to, closures of borders, schools, mosques, and Umrah (the minor pilgrimage to Mecca, which can be undertaken any time of the year); cessation of international flights; mandatory quarantine periods for returning travelers; workplace closures, with individuals working from home (apart from essential workers); and partial to complete lockdowns [8]. Digital health measures were also implemented and effectively utilized during the pandemic [9,10]. As of March 25, 2020, the MOH designated 25 hospitals with 80,000 hospital beds and 8000 intensive care unit beds for the treatment of COVID-19 cases. An additional 2200 beds were allocated for the isolation of suspected and quarantined cases [11]. Saudi Arabia has robust preparedness and response capabilities that have been strengthened through prior experience with the Middle East respiratory syndrome coronavirus and decades of planning and managing religious mass gatherings of Hajj (the annual pilgrimage to Mecca) and Umrah, which can serve as a model for other countries in the region.

    In public health emergencies such as the COVID-19 pandemic, effective communication is crucial for informing the public about disease situation updates, motivating them to adopt preventive measures, and reassuring them that the government is in control of the outbreak [12-14]. Such communication requires timely dissemination of accurate and reliable information. Traditionally, governments and public health authorities have relied on websites, print media, and television as the main platforms for disseminating outbreak-related information to the public. However, the evolution of digital communications technologies such as social media has facilitated increased sharing of information for both public health authorities and the general public.

    In recent years, social media has developed rapidly. Both individuals and health care organizations are using these platforms increasingly to communicate and share information [15]. Social media facilitates 2-way communication and direct engagement with audiences. Safko and Brake [16] define social media as “activities, practices, and behaviors among communities of people who gather online to share information, knowledge, and opinions using conversational media.” Today, there are over 3.8 billion active social media users worldwide across many different platforms [17]. In the field of health education and promotion, the use of social media has established its effectiveness by providing access to information, delivering health campaigns, and offering social support [18]. Many government agencies and public health organizations (eg, the WHO, the Centers for Disease Control and Prevention [CDC], and other local health departments) have adopted social media to enhance their communication with the public [19].

    Social media can serve as a useful tool to relay outbreak-related updates and critical information effectively to the public. Existing research suggests that people often turn to social media for information during infectious disease outbreaks, which can influence their decision-making and subsequent behaviors [19]. The WHO calls for more proactive use of social media to disseminate health messages to journalists, physicians, and the general public, particularly to counteract misinformation regarding infectious diseases [20].

    Several studies have investigated the use of social media platforms such as Twitter and Facebook during infectious disease outbreaks. For example, Chen et al [21] studied the temporal variability in the CDC’s response during different stages of the Zika epidemic and public engagement on Twitter. They reported that the CDC was more active in the early warning stages of the Zika epidemic and successfully gained public attention, particularly in the first quarter of 2016. However, when the number of Zika cases increased sharply in the second and third quarters of 2016, the CDC’s efforts on Twitter decreased substantially.

    Lwin et al [22] examined the strategic use of Facebook in communicating the Zika epidemic by three main Singapore health authorities: the National Environment Agency, the Health Promotion Board, and the MOH. The researchers found that Facebook was used strategically for Zika-related communication. They also found that preparedness messages (eg, posts mentioning responders and providing recommendations to reduce harm) may have been the most effective, as evidenced by greater levels of public engagement.

    Guidry et al [23] examined Ebola-related posts on Instagram and Twitter from three key health organizations: the CDC, the WHO, and Médecins Sans Frontières (ie, Doctors Without Borders). They found Instagram to be a particularly useful platform for communicating with the public during crises. It was further suggested that social media messaging is more effective when it is utilized by health organizations with which the public is already familiar, and when it is based on strategic use of risk communication principles.

    A recent study by Raamkumar [24] examined the use of Facebook for COVID-19–related outreach by public health authorities in Singapore, the United States, and England, and the corresponding public response to these efforts. They reported that the Singapore MOH was the most active in terms of posting frequency, while the CDC elicited the most responses. Furthermore, they reported that posts on preventive and safety measures and situation updates were the most frequently employed by public health authorities in these 3 countries.

    Theoretical Framework

    Crisis events such as the COVID-19 pandemic requires unique health communication and education strategies in which public health authorities must meet the public needs for information [14]. Theories suggests that the public has various information needs at different stages of a crisis [25]. The Crisis and Emergency Risk Communication (CERC) model serves as a useful tool to guide authorities’ communication strategies during different stages of a crisis. It specifies a broad set of communication activities that vary throughout the life cycle of the crisis. The CERC model was originally developed by the CDC after the 2001 anthrax attacks and the events of September 11, 2001, in the United States [26,27]. It is an integrated model that draws elements from risk communication theories (persuading individuals to take action to limit risks), crisis communication theories (responding to the public’s immediate need for information), and theories of health communication [27].

    The CERC model describes five general stages of a crisis: precrisis, initial event, maintenance, resolution, and evaluation. For each stage, a set of recommended communication activities is also described. According to the model, specific and distinct communication activities should be carried out in each stage.

    The first stage of the CERC model is the precrisis period. In this stage, the crisis has yet to occur. Communication messages at this stage should focus on risk information, warnings, and preparation.

    The second stage of the model is the initial event, when the crisis actually occurs. This stage is initiated by a clear trigger event that signals the beginning of a crisis. Communication messages in this stage should focus on reducing public uncertainty by providing timely updates regarding the crisis, messages of self-efficacy, and reassurance from authority-initiated measures.

    The third stage, or maintenance stage, begins when “most or all of the direct harm is contained, and the intensity of the crisis begins to subside” [28]. It further echoes many of the communication activities from earlier stages, including uncertainty reduction, reassurance, and self-efficacy messages.

    The fourth stage is the resolution stage, in which the crisis continues to wind down and new understandings of risk emerge. Communication messages at this stage involve updates about ongoing resolutions, discussions about causes, and new understandings of risk.

    The final stage is the evaluation stage. This stage occurs when the crisis itself is over. Communication during this stage should focus on assessing the adequacy and efficacy of the response and reaching a consensus on the lessons learned from the crisis [26,28].

    Objective

    With over 15 million Twitter users in Saudi Arabia [29], the microblogging and social media platform Twitter presents an opportunity to examine the Saudi MOH’s use of social media in crisis communication during pandemics. According to a recent national survey, approximately 78% of respondents reported the MOH as their main source of information about COVID-19 [30]. The objective of this study is to investigate the use of Twitter by the MOH and the associated public engagement during different stages of the COVID-19 pandemic in Saudi Arabia.


    Data Collection

    All tweets from the Saudi MOH (@SaudiMOH) posted between January 1 and August 31, 2020, were collected and included in the study. The tweets were collected on September 30, 2020, via the GET statuses/user_timeline endpoint of Twitter’s application programming interface [31] by using the python library Tweepy [32]. For each tweet, the following data were collected: tweet ID, tweet text (body), number of likes, number of retweets, and date posted. Only original tweets, rather than retweets, were considered. Tweets written in languages other than Arabic or English were removed to avoid misinterpretation. Tweets unrelated to COVID-19 were manually excluded by scanning the content of the tweets. Daily confirmed case counts of COVID-19 in Saudi Arabia were obtained from Our World in Data [33].

    Crisis Stages

    The various stages of the CERC model have not been clearly defined or operationalized within the context of infectious disease outbreaks. The CERC model assumes that “crises will develop in largely predictable and systematic ways” [26]. However, infectious disease outbreaks such as the current COVID-19 pandemic may last for months or even years, without clear boundaries (compared to other crises, such as extreme weather events) [34,35]. In this study, outbreak stages were determined on the basis of the CERC model and specific events of the COVID-19 outbreak in Saudi Arabia.

    The precrisis stage was determined to span from January 1 to March 1, 2020, when the outbreak began in China and some European countries, but when no cases had yet been identified in Saudi Arabia.

    The initial event stage was determined to last from March 2 to June 20, 2020. The first confirmed case of COVID-19 in Saudi Arabia was reported on March 2. During this period, there were 157,600 confirmed cases of COVID-19 in the country.

    The maintenance stage was determined to span from June 21 to August 31, 2020. On June 21, 2020, Saudi Arabian authorities lifted the nationwide curfew and allowed the resumption of all activities, except in Mecca.

    Content Analysis and Coding Categories

    This study used a content analysis approach to investigate the Saudi MOH’s communication on Twitter regarding the COVID-19 pandemic. Berg [36] explained that content analysis is a “careful, detailed, systematic examination and interpretation of a particular body of material in an effort to identify patterns, themes, biases, and meanings.” COVID-19–related tweets were coded using a codebook adapted from one that was developed on the basis of the CERC framework [22]. The codebook consisted of the following categories: (1) risk messages, including tweets containing information about the disease, its transmission mechanisms, and its symptoms; (2) warnings, including tweets highlighting risk factors and dangers associated with COVID-19; (3) preparations, including tweets mentioning responders and providing response recommendations and advice; (4) uncertainty reduction, including tweets containing information about case reports and providing the public with reliable sources of information; (5) efficacy, including tweets containing information about personal preventive measures and highlighting the common responsibility for disease prevention; and (6) reassurance, including tweets that calmed the public by providing information about government interventions and expressing gratitude and regards to the health staff and the public. Through initial scanning of the tweets, digital responses were found to be frequently mentioned in the MOH tweets. Given the importance of digital health during the COVID-19 pandemic, an additional category called “digital health responses” was introduced. Messages of this category pertained to tweets promoting digital health services, ranging from digital screening to surveillance, contact tracing, and follow-up apps. The codebook is presented in Multimedia Appendix 1.

    Each tweet was primarily categorized on the basis of the content within its 280 characters. If the content was not clear, linked visuals such as photographs, videos, and other media were analyzed. It was beyond the scope of this study to analyze the content of hyperlinks. The content analysis of the MOH’s tweets was conducted using Excel (version 16.43, Microsoft Inc), which was later imported into SPSS (version 27, IBM Corp) for statistical analysis.

    Intercoder Reliability

    Intercoder reliability was established by 2 independent coders (FH and SD). Each author independently coded a randomly selected subsample of 122 (10%) tweets. This meets the Neuendorf recommendation of coding 10%-20% of the total sample for reliability [37]. Reliability was assessed using the ReCal statistical program with the Cohen κ statistic [38]. The κ values for all categories were greater than 0.8, which indicated “almost perfect” agreement, except for the coding category of “Responders,” which had a κ value of 0.545, indicating “moderate agreement” [39]. Coding discrepancies for the “Responders” category were resolved through discussion. Once intercoder reliability was established, the first coder coded the remaining tweets.

    Statistical Analysis

    Statistical analysis was performed using SPSS (version 27) [40]. Counts and percentages were used to summarize categorical variables. The median was used to summarize the distribution of continuous variables owing to the skewed nature of those included. The chi-square test of independence was used to assess the associations between outbreak stages and message types. Significant chi-square outcomes were further subjected to multiple post hoc Z-tests to compare each pair of outbreak stages. P values were adjusted for the false discovery rate by using the Benjamini-Hochberg adjustment.

    The nonparametric Kruskal–Wallis test was used to compare the distribution of likes and retweets (engagement indicators) between outbreak stages. The Mann–Whitney U test was used to test differences in engagement between tweets including and those not including different message types.

    A negative binomial regression analysis was used to examine associations among tweet content, media type, crisis stage, and public engagement. Negative binomial regression was used as engagement variables demonstrated positive skew and overdispersion. The incidence rate ratio (IRR) was calculated as the exponent of the regression coefficients. All statistical tests were performed at a significance level of .05.

    Ethics Approval

    Ethics approval was not required for this study as the study did not involve any human subjects. All data analyzed in this study were publicly available and collected from a governmental public Twitter account.

    Data Availability

    The data that support our findings are available on request from the corresponding author.


    Results Overview

    The MOH posted a total of 1393 original tweets (an average of 5.85 tweets per day) during the study period. Overall, 1293 (92.8%) tweets were related to COVID-19, of which 1217 were included in the analysis. The other tweets (n=76) were removed because they were in languages other than Arabic or English. The results are presented in three sections: the MOH response to COVID-19 on Twitter across stages, message types across stages, and public engagement with tweets from the MOH.

    MOH Response to COVID-19 on Twitter Across Stages of the COVID-19 Outbreak

    Confirmed and reported COVID-19 cases were plotted in relation to COVID-19–related tweets posted by the MOH (Figure 1). The first COVID-19–related tweet was posted on January 21, 2020. Overall, 79 (6.5%) tweets were posted during the precrisis stage, when the outbreak began in China and some European countries, but when no cases had yet been identified in Saudi Arabia (2.03 tweets per day on average). On March 2, 2020, the MOH confirmed the first COVID-19 case in Saudi Arabia, which signaled the start of the crisis. COVID-19–related tweets were consistently posted as the number of cases increased, with an average of 7.23 daily tweets. As of June 20, 2020, the MOH had posted 802 tweets (66% of the total), which is a >3-fold increase in its average daily tweets compared to the precrisis stage. On June 21, 2020, the country lifted curfew restrictions and resumed all economic and commercial activities [41]. From that date until the end of August 2020, the MOH continued to provide ongoing information regarding COVID-19, with an average of 4.67 daily tweets (n=336, 27.6%).

    ‎
    Figure 1. The Saudi Ministry of Health's Twitter communication in relation to confirmed COVID-19 cases (January 1 to August 31, 2020). MOH: Ministry of Health.
    View this figure

    Message Types Across Stages of the COVID-19 Outbreak

    Message types across stages of the outbreak are summarized in Table 1. Of the 1217 tweets, nearly half (49.47%) contained uncertainty reduction information, 28.35% contained efficacy information, and one-fifth (21.53%) of all tweets contained reassuring information.

    Tweets about warning messages accounted for a low proportion of all tweets in the precrisis stage (2.5%), which significantly increased during the initial stage (χ22=6.2; P=.046) (Table 1 and Multimedia Appendix 2). Tweets about preparation messages accounted for a low proportion of tweets in the precrisis stage (11.4%) and the initial stage (11.2%), and the proportion increased significantly in the maintenance stage (22.3%; χ22=24.4; P<.001). Conversely, the percentage of reassurance tweets peaked in the precrisis stage (67.1%) and significantly decreased in later stages (χ22=103.9; P<.001). The frequency of efficacy tweets was higher in the initial stage (34.4%) than in the precrisis (15.2%) and maintenance (16.9%) stages (χ22=42.7; P<.001). Lastly, tweets promoting digital health services increased significantly in frequency, from 8.6% in the initial stage to 16.7% in the maintenance stage (χ22=26.4; P<.001).

    Table 1. Categories of message types across outbreak stages in Saudi Arabia (January 1 to August 31, 2020).
    Message typeaPrecrisis stage (n=79), n (%)Initial event stage (n=802), n (%)Maintenance stage (n=336), n (%)Overall (n=1217), n (%)P valueχ2 (df=2)
    Risk messages5 (6.3)62 (7.7)21 (6.3)88 (7.2).650.876

    		Disease information4 (5.1)24 (3.0)8 (2.4)36 (3.0).451.613

    		Symptoms3 (4.8)44 (5.5)15 (4.5)62 (5.1).670.805
    Warnings2 (2.5)91 (11.3)33 (9.8)126 (10.4).0466.162

    		Risk factor2 (2.5)40 (5.0)29 (8.6)71 (5.8).037.399

    		Danger0 (0.0)53 (6.6)4 (1.2)57 (4.7)<.00119.722
    Preparations9 (11.4)90 (11.2)75 (22.3)174 (14.3)<.00124.390

    		Responders8 (10.1)27 (3.4)11 (3.3)46 (3.8).0099.363

    		Recommendations1 (1.3)64 (8.0)64 (19.0)129 (10.6)<.00138.376
    Uncertainty reduction41 (51.9)382 (47.6)179 (53.3)602 (49.5).203.216

    		Case report30 (38.0)157 (19.6)72 (21.4)259 (21.3).00114.538

    		Information resources12 (15.2)225 (28.1)107 (31.8)344 (28.3).018.802
    Efficacy12 (15.2)276 (34.4)57 (17.0)345 (28.3)<.00142.699

    		Personal prevention12 (15.2)226 (28.2)51 (15.2)289 (23.7)<.00125.520

    		Common responsibility0 (0.0)86 (10.7)9 (2.7)95 (7.8)<.00128.447
    Reassurance53 (67.1)145 (18.1)64 (19.0)262 (21.5)<.001103.938

    		Calming51 (64.6)91 (11.3)41 (12.2)183 (15)<.001162.297

    		Thanks and regards0 (0.0)41 (5.1)23 (6.8)64 (5.3).0476.117

    		Government interventions4 (5.1)57 (7.1)26 (7.7)87 (7.1).710.696
    Digital health responses0 (0.0)69 (8.6)56 (16.7)125 (10.3)<.00126.375

    aA tweet can have more than 1 category.

    Public Engagement With MOH Tweets

    Figure 2 demonstrates public engagement (represented by frequencies of likes and retweets) in relation to MOH tweets. Public engagement was not accurately aligned with the development of the COVID-19 outbreak in the country; rather, the public was most engaged when the initial cases appeared in March 2020.

    Since engagement variables were not normally distributed, median values (rather than mean values) were used for statistical comparisons. MOH tweets were associated with median values of 819 likes and 603 retweets. A Kruskal–Wallis H test revealed significant differences in the frequencies of likes (χ22=70.344; P<.001) and retweets (χ22=59.764; P<.001) among different outbreak stages. The Mann–Whitney U test was used for post hoc comparisons of average ranks. Post hoc pairwise comparisons (Multimedia Appendix 3) revealed that the distribution of retweets was not significantly different between the precrisis and maintenance stages (Z=–2.14; P=.96).

    ‎
    Figure 2. Public engagement in relation to the Saudi Ministry of Health's tweets (January 1 to August 31, 2020). The left y-axis shows the daily number of tweets. The right y-axis shows the number of likes and retweets (in thousands). MOH: Ministry of Health.
    View this figure

    Median public engagement levels across different stages of the outbreak and message types are summarized in Table 2. Overall, the public engaged most with tweets from the MOH during the initial stage, with median values of 974 likes and 753 retweets. Further analysis using Mann–Whitney U tests (Table 3) revealed that both like and retweet frequencies during the initial stage were significantly higher for tweets that provided uncertainty reduction messages (Z=–3.133; P=.002) and reassurance messages (Z=–5.843; P<.001) than those that did not. Tweets containing risk messages at this stage received fewer likes (Z=–4.219; P<.001) and retweets (Z=–4.252; P<.001) than those that did not. In contrast, tweets containing risk messages during the precrisis stage received more likes (Z=–2.034; P=.04) and retweets (Z=–2.15; P=.03) than those that did not. Among tweets in the maintenance stage, the frequencies of both likes (Z=–3.708; P<.001) and retweets (Z=–3.605; P<.001) were significantly higher for tweets that contained uncertainty reduction information than for those that did not. Conversely, frequencies of both likes (Z=–4.534; P<.001) and retweets (Z=–4.547; P<.001) were significantly lower for tweets that promoted digital health services than for those that did not.

    Table 2. Median public engagement across outbreak stages and message types (January 1 to August 31, 2020).
    Message typeaLikesRetweets

    		All stagesPrecrisis stageInitial event stageMaintenance stageAll stagesPrecrisis stageInitial event stageMaintenance stage
    All819.0437.0974.0582.0603.0473.0753.0351.0
    Risk messages598.5686.0475.5630.0409.5766.0385.5381.0
    Warnings785.01678.0913.0513.0614.51801.0710.0335.0
    Preparations594.5343.0974.0464.0397.5438.0708.0273.0
    Uncertainty reduction841.5389.01025.0678.0658.5438.0770.0409.0
    Efficacy852.0560.0950.5590.0683.0623.5755.5422.0
    Reassurance982.0418.01629.0568.5740.0401.01198.0316.0
    Digital health responses650.00.0899.0452.0444.00.0620.0273.5

    aValues presented are medians.

    Table 3. Median public engagement for tweets with or without message type at different stages of the COVID-19 outbreak in Saudi Arabia (January 1 to August 31, 2020).
    VariableaMedian presentMedian absentU valueZ-valueP value
    Precrisis

    		Risk messages





    		Retweets766.00432.0078.000–2.154.03

    		Likes686.00403.5084.000–2.034.04
    Initial event

    		Risk messages





    		Retweets385.50781.5015490.000–4.252<.001

    		Likes974.501008.0015547.500–4.219<.001

    		Uncertainty reduction





    		Retweets770.00723.5070667.500–2.915.004

    		Likes1025.00964.5069956.000–3.133.002

    		Reassurance


    		Retweets1198.00659.0034581.000–5.169<.001

    		Likes1629.00856.0032880.500–5.843<.001
    Maintenance

    		Uncertainty reduction





    		Retweets409.00317.5010781.500–3.681<.001

    		Likes678.50524.5010680.000–3.795<.001

    		Digital health responses


    		Retweets273.50404.504823.000–4.547<.001

    		Likes452.00643.504831.000–4.534<.001

    aOnly significant variables are reported.

    Negative binomial regression outcomes are summarized in Table 4 (complete models are provided in Multimedia Appendix 4). The model was significantly better than the null model, which indicated that the variables (as a set) predicted the number of likes (n=1217; χ212=458.627; P<.001). The negative binomial regression model for retweets (Table 4) was also significantly better than the null model (n=1217; χ212=575.495; P<.001). The Wald test revealed that all of predictor variables were significant (P<.05), except for risk messages and preparations.

    Our results indicate that the use of hashtags was significantly associated with higher levels of engagement (likes: IRR=2.470; P<.001; retweets: IRR=2.813; P<.001), whereas the use of hyperlinks was significantly associated with lower levels of engagement (likes: IRR=0.839; P=.045; retweets: IRR=0.727; P<.001). Compared to text-only content, the use of photographs and videos was associated with significantly lower numbers of likes and retweets (photographs: likes, IRR=0.530; P<.001; retweets, IRR=0.476; P<.001; videos: likes, IRR=0.698; P=.006; retweets, IRR=0.576; P<.006).

    With respect to the impact of content type on public engagement, tweets with content related to warnings (likes: IRR=1.334; P=.005; retweets: IRR=1.544; P<.001), uncertainty reduction (likes: IRR=2.210; P<.001; retweets: IRR=2.197; P<.001), and reassurance (likes: IRR=1.551; P<.001; retweets: IRR=1.517; P<.001) were significantly associated with higher levels of engagement.

    Regarding crisis stages, tweets posted during the initial and maintenance stages were significantly associated with higher levels of engagement than those posted during the precrisis stage (initial stage: likes, IRR=2.931; P<.001; retweets, IRR=2.471; P<.001; maintenance stage: likes, IRR=2.355; P<.001; retweets, IRR=1.623; P<.001).

    Table 4. Associations of tweet content, media type, and crisis stage with public engagement (January 1 to August 31, 2020).
    VariablesLikesRetweets

    		Incidence rate ratio95% CIP valueIncidence rate ratio95% CIP value
    Intercept320.737227.465-452.255<.001325.265231.247-457.509<.001
    Hashtags2.4702.172-2.810<.0012.8132.469-3.206<.001
    Hyperlinks0.8390.707-0.996.0450.7270.611-0.865<.001
    Media type

    		Text onlyReferenceN/AaN/AReferenceN/AN/A

    		Photographs0.5300.428-0.656<.0010.4760.384-0.591<.001

    		Videos0.6980.540-0.901.0060.5760.445-0.745<.001
    Message type

    		Risk messages1.1770.934-1.482.1671.0880.863-1.371.48

    		Warnings1.3341.089-1.634.0051.5441.261-1.891<.001

    		Preparations1.0070.836-1.212.9441.0080.838-1.213.93

    		Uncertainty reduction2.2101.882-2.595<.0012.1971.870-2.583<.001

    		Efficacy1.0960.926-1.297.2881.2001.014-1.420.03

    		Reassurance1.5511.320-1.821<.0011.5171.296-1.776<.001
    Crisis stage

    		PrecrisisReferenceN/AN/AReferenceN/AN/A

    		Initial event2.9312.309-3.721<.0012.4711.945-3.141<.001

    		Maintenance2.3551.825-3.039<.0011.6231.256-2.096.001

    aN/A: not applicable.


    Principal Findings

    This study shows that Twitter was used as a real-time communication channel to share, communicate, and disseminate information during the COVID-19 pandemic. Overall, the MOH’s response to COVID-19 on Twitter was aligned with the developments of the outbreak in Saudi Arabia. During the precrisis and maintenance stages, the MOH’s use of Twitter was partially consistent with the CERC model. While in the initial stage, the MOH’s communication was in line with the CERC model. The results of public engagement showed that the levels of engagement were different as the pandemic evolved. The tweets in the initial stage elicited the most engagement by far.

    The number of tweets pertaining to the COVID-19 pandemic was relatively low (n=79) in the precrisis stage. However, when COVID-19 emerged in Saudi Arabia on March 2, 2020, and began to spread throughout the country, the MOH increased its Twitter activity. The MOH posted most of its COVID-19–related tweets (n=802, 65.89%) during this initial stage. This likely corresponded with the public’s need for information, given that individuals increasingly use social media to seek information during crises [34,42]. The MOH regularly posted tweets regarding COVID-19 (n=336) throughout the maintenance stage and until the end of the study period.

    The results concerning public engagement showed that the tweets in the initial stage received the most engagement from the public. This result was expected, considering the lockdowns and curfew restrictions imposed during this stage. Recent studies have even reported an increase in the usage of the internet and social media during COVID-19 lockdowns [43].

    The MOH’s use of Twitter was partially consistent with the CERC model. During precrisis, the CERC model suggests that communication should focus on risk information, warnings, and preparations. This is because during the precrisis stage, the public tends to seek information regarding the nature of the risk itself. However, our results show that a large proportion of the MOH’s tweets in this stage included reassurance (67.1%) or uncertainty reduction (51.9%), while those including risk messages and warnings represented only 8.8% of all tweets.

    Agwa [44], in her study of the Egyptian MOH’s use of Facebook during the COVID-19 pandemic, also observed a lack of risk information during early stages of the pandemic. One potential explanation for this finding is the novelty and scientific uncertainty associated with COVID-19 [45,46].

    During the initial stage, the CERC model suggests that communication messages should focus on reducing public uncertainty and providing messages regarding efficacy and reassurance. Consistent with the model, the MOH reduced uncertainty by updating case reports, holding press conferences, and providing information sources that accounted for 47.6% of tweets. The MOH tweets also emphasized efficacy (34.4%) by highlighting common responsibility and personal prevention measures to limit the spread of SARS-CoV-2 while providing reassurance (18.1%). The MOH’s communication during this stage was in line with the CERC model.

    As a crisis continues into the maintenance stage, the CERC model requires ongoing communication of uncertainty reduction and reassurance. Additional efficacy messages inform members of the public about the expected course of action at this stage.

    In accordance with the model, the MOH’s tweets during this stage provided information sources to reduce uncertainty in addition to efficacy and reassurance information. However, the results showed that a considerable proportion (22.3%) of the MOH’s tweets in this stage included preparation messages (mostly recommendations). This could potentially be part of the MOH’s efforts to prepare the public for the “new normal,” especially as the country lifted its nationwide curfew and began its gradual reopening during this stage. While the CERC model suggests that health communicators should offer reassurance in the maintenance stage, crisis communication experts do not recommend that messages be overly reassuring, as such information may reduce an authority’s credibility [47]. This is particularly the case in unexpected and unpredictable events such as the COVID-19 pandemic. In addition, it should be noted that some infectious disease outbreaks and epidemics commonly become chronic crises that develop into crisis stages for longer periods, making it difficult to make precise predictions [26].

    Most importantly, our findings indicate that different types of messages received different levels of engagement as the outbreak evolved. In the precrisis stage, the public showed a high level of interest in warnings and risk messages, as indicated by a high level of engagement. Based on this finding, it may be inferred that the public wanted to understand the full scope of the risk. In the initial stage, members of the public engaged more with certain messages (such as uncertainty reduction and reassurance) than others; indicating their simultaneous need for increased understanding and reduced anxiety regarding the outbreak. During the maintenance stage, the public also showed a high level of interest in information related to uncertainty reduction, which indicates that they may have still felt uncertain during this stage.

    Our findings also identified a number of factors associated with greater public engagement during the pandemic. First, our results show that the use of hyperlinks was negatively associated with public engagement. This further supports the inferences by Chung [48] that hyperlinks increase the complexity of a message by requiring an extra action by the audience, thereby reducing engagement. Another potential explanation is that hyperlinks direct people to another webpage, at which point they may forget about the original message. Second, in contrast with previous reports supporting the positive effect of media on public engagement [49-52], we found that the inclusion of multimedia content (eg, photographs and videos) was negatively associated with public engagement. This finding is similar to that of Chen et al [53], who also found that media richness was negatively associated with public engagement with government social media during the COVID-19 pandemic. This discrepancy can be attributed to the differences in the events examined, as most studies supporting the positive effect were conducted in noncrisis situations [53].

    The content of the tweets was also significantly associated with public engagement, where tweets related to uncertainty reduction, public reassurance, and warnings received higher levels of engagement. This is consistent with the results of Tang et al [54], who examined the public health agencies’ tweets in Texas, where tweets that provided information about COVID-19 or described the government’s actions in containing the spread of COVID-19 were found to be more likely to be retweeted. This suggests that as the CERC model indicates, people need this type of information during a public health crisis.

    Limitations

    There are several limitations and future considerations of this study. First, while this study focused on Twitter, the MOH used other social media platforms (such as Facebook and Instagram) during the COVID-19 pandemic. Twitter is a microblogging platform that allows users to post short messages that contain up to 280 characters. Facebook allows for much longer posts than Twitter, while Instagram is centered on images rather than text. Since social media platforms vary greatly in their characteristics, different communication strategies may be adopted and employed. Future studies should consider focusing on the same topic on Facebook, Instagram, and other popular social media platforms.

    Second, given that the pandemic is still ongoing, this study focused on the first 3 stages of the crisis (precrisis, initial, and maintenance stages) and did not examine the resolution or evaluation stage. Future studies should expand the scope of the analysis to provide a more comprehensive description of the MOH’s crisis communication on Twitter.

    Only tweets written in Arabic and English were included in the analysis. In addition, this study did not attempt to analyze public replies to the MOH’s tweets. Further studies are needed to examine the content of the public’s replies to understand their responses and opinions regarding tweets from the MOH. A final limitation is the time interval between the posting date of a tweet and the date of data collection. Since older tweets may take a longer time to accumulate engagement, future studies should consider such temporal effects. Future studies should also incorporate more appropriate measures of engagement, beyond simply the numbers of likes and retweets.

    Conclusions

    The COVID-19 pandemic is an extreme crisis and has generated significant challenges for governments. Effective communication with the public is of crucial importance. This study provided some insight into the Saudi MOH’s outbreak communication strategy. Our findings identified differences in MOH communication practices during different stages of the COVID-19 outbreak in Saudi Arabia, in terms of both types of message content and levels of public engagement. Uncertainty reduction, efficacy, and reassurance were the most common types of messages in MOH tweets. Our results provide several implications for crisis communication by researchers, governments, health organizations, and practitioners to engage their external public. Governments and health authorities should consider the public’s information needs to promote their engagement; this, in turn, could raise the public’s awareness of a health crisis. Effective communication during disease outbreaks and other public health emergencies has the potential to change outcomes and save lives.

    Acknowledgments

    The authors gratefully acknowledge King Abdullah International Medical Research Center and King Saud bin Abdulaziz University for Health Sciences for supporting and funding the publication of this paper.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Codebook.

    DOCX File , 19 KB
    ‎
    Multimedia Appendix 2

    Categories of message types by outbreak stages, post-hoc pairwise comparisons.

    DOCX File , 17 KB
    ‎
    Multimedia Appendix 3

    Public engagement across outbreak stages, post-hoc pairwise comparisons.

    DOCX File , 14 KB
    ‎
    Multimedia Appendix 4

    Negative binomial regression results for associations of tweet content, media type, and the crisis stage with public engagement.

    DOCX File , 21 KB

    References

    1. Fehr AR, Perlman S. Coronaviruses: an overview of their replication and pathogenesis. Methods Mol Biol 2015;1282:1-23 [FREE Full text] [CrossRef] [Medline]
    2. Dong E, Du H, Gardner L. An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect Dis 2020 May;20(5):533-534 [FREE Full text] [CrossRef] [Medline]
    3. Giovanetti M, Benvenuto D, Angeletti S, Ciccozzi M. The first two cases of 2019-nCoV in Italy: Where they come from? J Med Virol 2020 May;92(5):518-521 [FREE Full text] [CrossRef] [Medline]
    4. Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, China Novel Coronavirus Investigating and Research Team. A Novel Coronavirus from Patients with Pneumonia in China, 2019. N Engl J Med 2020 Feb 20;382(8):727-733 [FREE Full text] [CrossRef] [Medline]
    5. Wu F, Zhao S, Yu B, Chen Y, Wang W, Song Z, et al. A new coronavirus associated with human respiratory disease in China. Nature 2020 Mar;579(7798):265-269 [FREE Full text] [CrossRef] [Medline]
    6. Population Estimates. General Authority for Statistics, Kingdom of Saudi Arabia.   URL: https://www.stats.gov.sa/en/43 [accessed 2021-04-12]
    7. Statistical Yearbook. Ministry of Health, Kingdom of Saudi Arabia.   URL: https://www.moh.gov.sa/en/Ministry/Statistics/book/Pages/default.aspx [accessed 2021-04-12]
    8. Algaissi AA, Alharbi NK, Hassanain M, Hashem AM. Preparedness and response to COVID-19 in Saudi Arabia: Building on MERS experience. J Infect Public Health 2020 Jun;13(6):834-838 [FREE Full text] [CrossRef] [Medline]
    9. Hassounah M, Raheel H, Alhefzi M. Digital Response During the COVID-19 Pandemic in Saudi Arabia. J Med Internet Res 2020 Sep 01;22(9):e19338 [FREE Full text] [CrossRef] [Medline]
    10. National Digital Transformation Semiannual Report. National Digital Transformation Unit. 2020.   URL: https://ndu.gov.sa/report/2020-bareport.pdf [accessed 2021-04-06]
    11. WHO, Saudi Arabia join forces to fight COVID-19 nationally, regionally and globally. World Health Organization. 2020 Mar 25.   URL: https:/​/www.​emro.who.int/​media/​news/​who-saudi-arabia-join-forces-to-fight-covid-19-nationally-regionally-and-globally.​html [accessed 2021-04-15]
    12. Jones SC, Waters L, Holland O, Bevins J, Iverson D. Developing pandemic communication strategies: Preparation without panic. J Bus Res 2010 Feb;63(2):126-132. [CrossRef]
    13. Zhao Y, Cheng S, Yu X, Xu H. Chinese Public's Attention to the COVID-19 Epidemic on Social Media: Observational Descriptive Study. J Med Internet Res 2020 May 04;22(5):e18825 [FREE Full text] [CrossRef] [Medline]
    14. Strekalova YA. Health Risk Information Engagement and Amplification on Social Media. Health Educ Behav 2017 Apr;44(2):332-339. [CrossRef] [Medline]
    15. Moorhead SA, Hazlett DE, Harrison L, Carroll JK, Irwin A, Hoving C. A new dimension of health care: systematic review of the uses, benefits, and limitations of social media for health communication. J Med Internet Res 2013 Apr 23;15(4):e85 [FREE Full text] [CrossRef] [Medline]
    16. Safko L, Brake DK. The Social Media Bible: Tactics, Tools, and Strategies for Business Success. Hoboken, NJ: Wiley; 2009.
    17. Kemp S. Digital 2020: Global Digital Overview. DataReportal. 2020 Jan 30.   URL: https://datareportal.com/reports/digital-2020-global-digital-overview [accessed 2020-12-28]
    18. Korda H, Itani Z. Harnessing social media for health promotion and behavior change. Health Promot Pract 2013 Jan;14(1):15-23. [CrossRef] [Medline]
    19. Tang L, Bie B, Park S, Zhi D. Social media and outbreaks of emerging infectious diseases: A systematic review of literature. Am J Infect Control 2018 Sep;46(9):962-972 [FREE Full text] [CrossRef] [Medline]
    20. Developing tools for strategic communication to the media on emerging infectious diseases (‎EID)‎ : report of an informal consultation, WHO-SEARO, New Delhi, 26 May 2011. World Health Organization.   URL: https://apps.who.int/iris/handle/10665/204966 [accessed 2021-05-12]
    21. Chen S, Xu Q, Buchenberger J, Bagavathi A, Fair G, Shaikh S, et al. Dynamics of Health Agency Response and Public Engagement in Public Health Emergency: A Case Study of CDC Tweeting Patterns During the 2016 Zika Epidemic. JMIR Public Health Surveill 2018 Nov 22;4(4):e10827 [FREE Full text] [CrossRef] [Medline]
    22. Lwin MO, Lu J, Sheldenkar A, Schulz PJ. Strategic Uses of Facebook in Zika Outbreak Communication: Implications for the Crisis and Emergency Risk Communication Model. Int J Environ Res Public Health 2018 Sep 10;15(9):1974 [FREE Full text] [CrossRef] [Medline]
    23. Guidry JP, Jin Y, Orr CA, Messner M, Meganck S. Ebola on Instagram and Twitter: How health organizations address the health crisis in their social media engagement. Public Relat Rev 2017 Sep;43(3):477-486. [CrossRef]
    24. Sesagiri Raamkumar A, Tan SG, Wee HL. Measuring the Outreach Efforts of Public Health Authorities and the Public Response on Facebook During the COVID-19 Pandemic in Early 2020: Cross-Country Comparison. J Med Internet Res 2020 May 19;22(5):e19334 [FREE Full text] [CrossRef] [Medline]
    25. Reynolds B, Quinn Crouse S. Effective communication during an influenza pandemic: the value of using a crisis and emergency risk communication framework. Health Promot Pract 2008 Oct;9(4 Suppl):13S-17S. [CrossRef] [Medline]
    26. Reynolds B, Seeger MW. Crisis and emergency risk communication as an integrative model. J Health Commun 2005;10(1):43-55. [CrossRef] [Medline]
    27. Veil S, Reynolds B, Sellnow TL, Seeger MW. CERC as a theoretical framework for research and practice. Health Promot Pract 2008 Oct;9(4 Suppl):26S-34S. [CrossRef] [Medline]
    28. Centers for Disease Control and Prevention.   URL: https://emergency.cdc.gov/cerc/ [accessed 2021-06-12]
    29. Leading countries based on number of Twitter users as of April 2020 (in Millions). Statista.   URL: https://www.statista.com/statistics/242606/number-of-active-twitter-users-in-selected-countries/ [accessed 2020-05-28]
    30. Aldarhami A, Bazaid AS, Althomali OW, Binsaleh NK. Public Perceptions and Commitment to Social Distancing "Staying-at-Home" During COVID-19 Pandemic: A National Survey in Saudi Arabia. Int J Gen Med 2020;13:677-686 [FREE Full text] [CrossRef] [Medline]
    31. Get Tweet timelines. Twitter.   URL: https:/​/developer.​twitter.com/​en/​docs/​twitter-api/​v1/​tweets/​timelines/​api-reference/​get-statuses-user_timeline [accessed 2021-03-10]
    32. Tweepy: An easy-to-use Python library for accessing the Twitter API. Tweepy.   URL: https://www.tweepy.org/ [accessed 2021-03-10]
    33. Ritchie H, Ortiz-Ospina E, Beltekian D, Mathieu E, Hasell J, Macdonald B, et al. Statistics and Research: Coronavirus (COVID-19) Cases. Our World in Data.   URL: https://ourworldindata.org/covid-cases [accessed 2021-07-05]
    34. Lachlan KA, Spence PR, Lin X, Najarian K, Del Greco M. Social media and crisis management: CERC, search strategies, and Twitter content. Comput Hum Behav 2016 Jan;54:647-652. [CrossRef]
    35. Panagiotopoulos P, Barnett J, Bigdeli AZ, Sams S. Social media in emergency management: Twitter as a tool for communicating risks to the public. Technol Forecast Soc Change 2016 Oct;111:86-96. [CrossRef]
    36. Berg B. Qualitative Research Methods for the Social Sciences. Boston, MA: Allyn and Bacon; 2001.
    37. Neuendorf K. The content analysis guidebook. Thousand Oaks, CA: Sage Publications; 2002.
    38. Freelon D. ReCal: Intercoder Reliability Calculation as a Web Service. Int J Internet Sci 2010;5(1):20-33 [FREE Full text] [CrossRef]
    39. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977 Mar;33(1):159-174. [Medline]
    40. IBM SPSS Statistics for Macintosh, version 27.0. Armonk, NY: IBM Corp.   URL: https://www.ibm.com/analytics/spss-statistics-software [accessed 2021-07-05]
    41. An Approval Issued to fully Lift the Curfew from 06 am on Sunday, Ban on Umrah, Visit, Int'l Flights to Continue. Saudi Press Agency.   URL: https://www.spa.gov.sa/viewfullstory.php?lang=en&newsid=2100088 [accessed 2021-05-12]
    42. Lachlan KA, Spence PR, Lin X. Expressions of risk awareness and concern through Twitter: On the utility of using the medium as an indication of audience needs. Comput Hum Behav 2014 Jun;35:554-559. [CrossRef]
    43. Donthu N, Gustafsson A. Effects of COVID-19 on business and research. J Bus Res 2020 Sep;117:284-289 [FREE Full text] [CrossRef] [Medline]
    44. Ajwa NA. Strategies for communicating health risks through social media against the Corona pandemic (analytical study on the official pages of the Egyptian Ministry of Health). J Mass Commun Res 2020 Aug 01;54(4):2433-2494. [CrossRef]
    45. Tang JW. COVID-19: interpreting scientific evidence - uncertainty, confusion and delays. BMC Infect Dis 2020 Sep 08;20(1):653 [FREE Full text] [CrossRef] [Medline]
    46. Han PKJ, Scharnetzki E, Scherer AM, Thorpe A, Lary C, Waterston LB, et al. Communicating Scientific Uncertainty About the COVID-19 Pandemic: Online Experimental Study of an Uncertainty-Normalizing Strategy. J Med Internet Res 2021 Apr 22;23(4):e27832 [FREE Full text] [CrossRef] [Medline]
    47. Seeger MW. Best Practices in Crisis Communication: An Expert Panel Process. J Appl Commun Res 2006 Aug;34(3):232-244. [CrossRef]
    48. Chung JE. Retweeting in health promotion: Analysis of tweets about Breast Cancer Awareness Month. Comput Hum Behav 2017 Sep;74:112-119. [CrossRef]
    49. Guidry JPD, Meganck SL, Lovari A, Messner M, Medina-Messner V, Sherman S, et al. Tweeting about #Diseases and #Publichealth: Communicating Global Health Issues across Nations. Health Commun 2020 Aug;35(9):1137-1145. [CrossRef] [Medline]
    50. Bhattacharya S, Srinivasan P, Polgreen P. Social media engagement analysis of U.S. Federal health agencies on Facebook. BMC Med Inform Decis Mak 2017 Apr 21;17(1):49 [FREE Full text] [CrossRef] [Medline]
    51. Siyam N, Alqaryouti O, Abdallah S. Mining government tweets to identify and predict citizens engagement. Technol Soc 2020 Feb;60:101211. [CrossRef]
    52. Bonsón E, Perea D, Bednárová M. Twitter as a tool for citizen engagement: An empirical study of the Andalusian municipalities. Gov Inf Q 2019 Jul;36(3):480-489. [CrossRef]
    53. Chen Q, Min C, Zhang W, Wang G, Ma X, Evans R. Unpacking the black box: How to promote citizen engagement through government social media during the COVID-19 crisis. Comput Human Behav 2020 Sep;110:106380 [FREE Full text] [CrossRef] [Medline]
    54. Tang L, Liu W, Thomas B, Tran HTN, Zou W, Zhang X, et al. Texas Public Agencies' Tweets and Public Engagement During the COVID-19 Pandemic: Natural Language Processing Approach. JMIR Public Health Surveill 2021 Apr 26;7(4):e26720 [FREE Full text] [CrossRef] [Medline]

    ‎
    CDC: Centers for Disease Control and Prevention
    CERC: Crisis and Emergency Risk Communication
    IRR: incidence rate ratio
    MOH: Ministry of Health
    WHO: World Health Organization

    Edited by T Sanchez; submitted 14.02.21; peer-reviewed by K Lachlan, M Pobiruchin, A Sudaryanto; comments to author 07.03.21; revised version received 29.04.21; accepted 11.06.21; published 12.07.21

    Copyright

    ©Fatimah Mohammed Alhassan, Sharifah Abdullah AlDossary. Originally published in JMIR Public Health and Surveillance (https://publichealth.jmir.org), 12.07.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on https://publichealth.jmir.org, as well as this copyright and license information must be included.