Communicable diseases remain the most common cause of death, disability, and illness in many developing countries [1]. The most critical step to prevent and control epidemics effectively is the timely detection of outbreaks, which depends on effective disease surveillance systems.

Electronic Diseases Early Warning System (eDEWS) is an electronic system for data collection, compilation, and analysis from the health facilities to detect outbreaks at an early stage and take necessary response measures to prevent or limit disease occurrence. This can minimize the morbidity and mortality owing to communicable diseases through the detection of potential outbreaks at their earliest possible stage and facilitate timely interventions [2,3]. eDEWS is based on a mobile app. It enables health workers to report in real time about public health threats such as acute watery diarrhea, polio (Acute Flaccid Paralysis), measles, meningitis, and viral hemorrhagic fevers, including Ebola. Through eDEWS, data are collected using software installed on mobile phone, tablets, or laptops and immediately sent to a central level for analysis and rapid response. With faster reporting, increase in the number of patients with diseases of public health concern can be spotted more quickly, enabling prompt follow-up action [4].

Surveillance systems in most of the Eastern Mediterranean countries are traditional and still depend on manual recording, which causes delay in reporting and also missing data. In response to the need for strengthening the existing surveillance systems and improving the speed and efficiency of data collection, analysis, and public health response, World Health Organization (WHO) in collaboration with the Ministries of Health decided to implement the eDEWS in several countries [3,5].

eDEWS was launched in Yemen in March 2013, at 4 governorates, Aden, Lahij, Abyan, and Taiz, in 100 health facilities for 16 priority infectious diseases [5-7].

The various surveillance activities are integrated into 1 system within the broader national health information to enhance the speed and efficiency of data collection, analysis, and public health response [3].

Since the beginning of the War in March 2015, limited access to health care services and a breakdown in safe water supply and sanitation services has triggered the spread of endemic diseases such as malaria and dengue fever, as well as acute diarrheal diseases.

In response to the need for electronic expansion of the existing public health surveillance system and improving the speed and efficiency of data collection, analysis, and public health response, WHO in collaboration with the Ministry of Public Health and Population has scaled up the system in all Yemen’s governorates, with sentinel sites expanded from 408 health facilities in 16 governorates to 1186 facilities in 23 governorates after the third expansion at the end of 2016 [3,8].

This study is the first evaluation study conducted in Sana’a governorate to identify the strengths and weaknesses of eDEWS, determine its usefulness, and assess its performance in terms of the system attributes, including simplicity, flexibility, data quality, acceptability, representativeness, timeliness, and stability.

The CDC’s updated guidelines [9] on the evaluation of public health surveillance were used to evaluate the eDEWS in Sana’a governorate during November to December 2016. Sana’a governorate was covered by eDEWS in 2015. Its population was 918,379 inhabitants, living in 16 districts. The health facilities were at 25 sites in 2015, and with expansion at the end of 2016, health facilities reached 149 sites. Only the first 25 sites were subjected to the evaluation as the expansion happened few months before this evaluation took place.

To conduct the evaluation, stakeholders from different levels (WHO and Ministry of Public Health & Population (MoPHP), eDEWS central, governorate, and health facility levels) were included. Stakeholders at the central level (national coordinator of eDEWS and staff, including data management officer, data follow-up officer, and the Information Technology (IT) officer), governorate level (coordinator), and health facility level (focal points) in 16 districts were included.

A total of 4 questionnaires were designed and used to collect data from participants at the 4 levels of eDEWS using face-to-face interview. Each questionnaire comprised items assessing the performance attributes of the system according to the activity of eDEWS at the 4 levels. The assessment of usefulness was limited to governorate, central, and high levels, whereas the assessments of timeliness and completeness were limited only to the central level. Usefulness was assessed by 8 items using 5-point Likert scale (1=strongly disagree, 2=disagree, 3=neutral, 4=agree, and 5=strongly agree) for governorate, central, and high levels. Qualitative and quantitative system attributes were assessed. The qualitative attributes included simplicity, flexibility, acceptability, representativeness, and stability. The quantitative attributes included data quality and timeliness. Weekly reports of eDEWS and available documents of the system were reviewed for completeness and timeliness and to describe the eDEWS surveillance system.

The mean percentage scores for usefulness and other performance attributes were calculated by dividing the sum of the items measuring each performance attribute by the maximum score. The scores were interpreted as excellent (>90%), good (80%-90%), average (60%-79.9%), poor (40%-59.9%), and very poor (<40%). The overall attribute score was calculated by summing all items over all attributes and dividing it by the maximum score. During the analysis, the attitude was considered as positive, if the responses were strongly agree or agree, whereas the attitude was considered as negative, if the responses were strongly disagree or disagree.

Institution approval for using the program data was obtained from the eDEWS program before beginning the evaluation. Confidentiality of the gathered information was assured by anonymity and using passwords on the computer. Participation was absolutely voluntary.

The eDEWS is a program in the Diseases Control and Surveillance Directorate. Figure 1 shows the mechanism of the report flow and feedback. eDEWS has 2 main components: (1) immediate alert component for diseases which should be reported within 24 hours after detection such as AFP/polio, cholera, and hemorrhagic fever and (2) weekly reporting component. Weekly data aggregated by the health facilities are reported through the hierarchy of administrative levels (eg, health facility sites, governorates, and central level). At the central level, the system is operated by eDEWS national coordinator, data management officer, IT officer, and data follow-up officer. The program has a governorate coordinator in each governorate and focal point in each health site.

At the governorate level, the governorate coordinator was asked to appraise the usefulness of eDEWS based on 6 items. The mean percentage was 93% for usefulness. He scored 5 for 5 items and 3 for 1 item. He strongly agreed that the main aim of eDEWS is to detect diseases and outbreaks at an early stage and that data are used to monitor the trends of disease, to estimate the morbidity and mortality for diseases, and to detect outbreaks for rapid action to be taken. However, receiving epidemiological bulletin on time was scored as neutral.

At the high and central levels, the overall score of usefulness was good (mean percentage=83%). All 6 participants strongly agreed that the main aim of eDEWS is the early detection of diseases and outbreaks and all agreed that the data of the system are used to detect morbidity and mortality. All except 1 participant (83.3%) agreed that the data of the system are used to show trends of communicable diseases. In addition, 4 out of 6 (67%) respondents strongly agreed and 2 (33%) agreed that the data are used to facilitate rapid action to be taken.

The governorate coordinator was asked to appraise the simplicity of eDEWS according to 13 items. A total of 8 items were scored 5, 1 item was scored 4, 2 items were scored 3, and 1 item was scored 2. The mean percentage of simplicity was 83%, on which the governorate coordinator strongly agreed that the data entry in the electronic system is easy and clear. In addition, he strongly agreed that the notification process and transferring of data to the central level are easy. The coordinator agreed that the data follow-up is necessary, and analysis of data is conducted with respect to person, place, and time. He also reported that he receives a feedback from the central level. However, he reported that the phone and internet are not always available at his site and training courses are not performed frequently.

The overall system performance was good (86%). The highest attribute score was 100% for representativeness and the lowest score was 70% for stability (Table 3).

All stakeholders at high and central levels were given a chance to express their opinions concerning the strengths and weaknesses of the system. All stakeholders reported that the main strength was the rapid detection and response to diseases and outbreaks. Half of them reported that the data had high accuracy owing to validation that passed through 4 levels and 50% reported that the processing and analysis of data automatically was one of the strengths. On the other hand, all stakeholders in high and central levels said that the main weakness was the network weakness in some remote areas aggravated by war. The director of the system at the central level said that “there is no operational cost for maintenance the electronic instruments and mobiles.”