Background: The Neonatal Tetanus Surveillance System (NTSS) in Yemen was established in 2009 to identify high-risk areas, determine trends, and evaluate elimination activities. Since its launch, the NTSS had never been evaluated.
Objective: This study aimed to assess the performance of NTSS and determine its strengths and weaknesses to recommend improvements.
Methods: The US Centers for Disease Control and Prevention (CDC) guidelines were used for evaluating the NTSS. Stakeholders at the central, district, and facility levels were interviewed to rate the attributes of the NTSS. The percentage scores for attributes were ranked as poor (<60%), average (≥60% to <80%) and good (≥80%).
Results: The overall usefulness score percentage was 38%, which indicates a poor performance. The performance of the NTSS was rated as average on flexibility (score percent: 68%) and acceptability (score percent: 64%) attributes and poor on stability (score percentage: 33%), simplicity (score percentage: 57%), and representativeness (score percentage: 39%) attributes. About 65% of investigation forms were completed within 48 hours of notification date. Data quality was poor, as 41% of the core variables were missing.
Conclusions: The overall performance of the NTSS was poor. Most of the system attributes require improvement, including stability, simplicity, quality of data, and completeness of investigation. To improve the performance of NTSS, the following are recommended: capacity building of staff (focal points), strengthening NTSS through technical support and government funding to ensure its sustainability, establishing electronic investigation forms for improving the system data quality, and expansion of NTSS coverage to include all private health care facilities.
Neonatal tetanus (NT) is a life-threatening vaccine-preventable disease and one of the most underreported diseases in many developing countries. It is caused by the toxin of Clostridium tetani, a ubiquitous, spore-forming, gram-positive bacillus found in high concentrations in soil and animal excrement. The disease usually occurs in rural settings with poor access to health facilities, characterized by generalized rigidity and convulsive spasms of skeletal muscles that usually involve the jaw and neck and then become generalized . Newborns can become infected through contaminated instruments used to cut the umbilical cord or by improper handling of the umbilical stump [ , ].
NT remains a major cause of infant and neonatal mortality in many developing countries, but it is preventable by immunization and/or assuring clean delivery and postdelivery practices . According to the World Health Organization (WHO), an NT case is defined as “a neonate with the normal ability to suck and cry during the first 2 days of life, between 3 to 28 days of age cannot suck normally and becomes stiff or has spasms” [ ]. Global elimination is defined as an annual rate of <1 case of NT per 1000 live births at the district level [ ].
Globally, good progress has been made in the last two decades. The number of reported NT cases worldwide declined by 90%, from 17,935 in 2000 to 1803 in 2018, and estimated NT deaths decreased by 85%, from 170,829 in 2000 to 25,000 in 2018 . This disease accounts for 1% of worldwide neonatal mortality, compared with 7% in 2000 [ ].
Despite the appreciable drop in the number of NT cases attributed to the global NT elimination strategies, NT still remains among the leading causes of death in 14 developing countries that have not yet eliminated NT . The Eastern Mediterranean Region (EMR) has been ranked at the fourth position among WHO regions in term of NT burden. In 2018, 181 (10%) of NT cases were reported from 6 countries in the region: Afghanistan, Saudi Arabia, Iraq, Morocco, Egypt, and Yemen [ ].
Neonatal Tetanus Surveillance in Yemen
Yemen is one of the 5 remaining countries in the EMR that have not achieved the global elimination target set by WHO . NT remains a public health problem in Yemen, where it is one of the major causes of neonatal mortality. In 2018, more than 64% of all NT cases in EMR were reported from Yemen. A total of 2069 NT cases were reported from 1980 to 2018 [ ].
Tetanus immunization has been included in the national expanded program of immunization in Yemen since 1977, and includes 5 doses of tetanus toxoid vaccine for women of child-bearing age. Despite the availability of an inexpensive and effective tetanus vaccine, Yemen is far behind in tetanus toxoid vaccine coverage. Over the last decade, 17% to 21% of women received at least 2 doses of the tetanus toxoid vaccine in Yemen .
The Neonatal Tetanus Surveillance System (NTSS) was established in Yemen in 2009, with the objectives to identify high-risk areas, determine trends of NT, and evaluate tetanus elimination activities. NTSS in Yemen is a passive surveillance system that relies on the identification of NT cases collected at the health facility level and then reported to the next higher level at a specified frequency (weekly), even if there are zero cases (referred to as zero reporting). In addition, all NT cases are reported through community-based surveillance (CBS) and the Electronic Disease Early Warning System (eIDEWS). Particularly in high-risk areas, the NTSS relies on CBS through traditional birth attendants, community leaders, traditional healers, or other community members who are sensitized to report NT cases and deaths to health authorities. The eIDEWS network of reporting sites includes both public and private health facilities.
High-quality NT surveillance is a key component of the NT elimination strategy, and its data are used to identify areas or subpopulations at high-risk for NT and guide effective public health response for NT elimination. However, since its launch in Yemen, NTSS had never been evaluated. Therefore, this study aimed to assess the level of usefulness and performance of the NTSS attributes, determine its strengths and weaknesses, and suggest recommendations for improvement.
A descriptive evaluation study was conducted to describe NTSS and its performance according to the Centers for Disease Control and Prevention (CDC) guideline for evaluating disease surveillance systems . The study was performed in the city of Sana’a from October to December 2018.
Study Population and Sampling
The study population involved 31 stakeholders: 5 from the central level, 3 from governorate level, 3 from districts level, and 20 health facility focal points. At the district and health facility levels, samples were selected using a multistage sampling method. Using simple random sampling, 30% (3/10) of the districts—Assabain, Ma’ain, and Assafi’yah—were selected. A total of 33% (20/60) of health facilities were selected by stratified random sampling distributed proportionally among the chosen districts according to the curative services (public and private) in each district. The selection of health facilities from each chosen district was based on simple random sampling.
Desk review of the main NTSS documents and literature was conducted to describe the system. In depth interviews were conducted with NTSS stakeholders at the central and governorate levels to measure usefulness, flexibility, and stability. A semistructured questionnaire was used to collect data from surveillance coordinators of governorates, districts, and focal points at health facility levels to evaluate simplicity and acceptability (). In addition, data registries from 2009 to 2018 were obtained to measure data quality, timeliness of investigation, completeness of investigation, and representativeness.
Performance of the System
The indicators of the level of usefulness, flexibility, and stability attributes were assessed using yes or no (yes = 1; no = 0) questions. The indicators of simplicity and acceptability were assessed using a 5-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = neutral, 4= agree, and 5 = strongly agree). Quality of data was assessed by measuring the percentages of missing variables. Timeliness of investigation was measured by the percentage of all suspected cases investigated within 48 hours of notification. Completeness of investigation was measured by the proportion of NT suspected cases reported that have been investigated. Representativeness was measured by the percentage of all public and private health facilities in Sana’a that were covered by NTSS.
Data Analysis and Interpretation
The usefulness, flexibility, and stability indicators were scored as 0 or 1, whereas the simplicity and acceptability indicators were scored from 1 to 5. For each indicator, the score percentage was calculated as the following:
The overall attribute score percentage was calculated as the following:
Each indicator and overall attribute score percentage was represented as score rank and interpreted as poor <60%, average ≥60% to <80%, or good ≥80%. Missing data were measured by selecting 1-year data to calculate the percentage of the missed core variables. The core variables included case identification, date of birth, sex, place of usual residence, date of illness onset, date of notification, date of investigation, symptoms in case definition, outcome (alive/dead), maternal vaccination history, place/type of delivery, tool used for cutting cord, and material applied to cord. For any case, if information on any of the core variables is missing, the investigation was considered inadequate.
Timeliness of investigation was calculated by dividing the number of suspected NT cases investigated within 48 hours of notification by the number of suspected NT cases investigated × 100. Completeness of investigation was calculated by dividing the number of NT case investigations by the number of suspected NT cases reported × 100. Representativeness was calculated by dividing the number of public and private health facilities in Sana’a that are covered by NTSS by the total number of public and private health facilities in Sana’a × 100. Epi Info (version 7.2, Division of Health Informatics and Surveillance, CDC) was used for data entry and analysis. Data were described using frequencies, score percentages, and score rank.
A total of 31 participants (61% [19/31] male and 39% [12/31] female) were interviewed from 4 levels: central, governorate, district, and health facility. The median of their ages was 42 years (range 24 to 65 years) and the median years of their experience in surveillance systems was 4.0 years (range 1 to 20 years).
Level of Usefulness
Based on 8 in-depth interviews with stakeholders at central and governorate levels, the overall usefulness score percentage was 38%, which indicates a poor performance. Only one usefulness indicator (“The system data are used to estimate NT magnitude, incidence, and mortality rates”) was ranked as good (). One of the respondents said “Unfortunately, NT is neglected by researchers and the Ministry of Public Health and Population.”
|The system data are used to estimate NTa magnitude, incidence, and mortality rates.||7||88||Good|
|The system data are used to monitor the trend of NT spread over time.||4||50||Poor|
|The system data are used to identify high-risk areas.||4||50||Poor|
|The system data are used to update and develop the national policy strategy for NT elimination.||1||13||Poor|
|The system data are used to assess the effect of interventions.||1||13||Poor|
|The system provides a basis for epidemiologic research.||1||13||Poor|
aNT: neonatal tetanus.
The overall score of flexibility was 68%, which reveals an average performance (). Three of the flexibility indicators (“The system can be adapted to accommodate new additional information” [eg, change in case definition], “The system can be adapted to increases in the reporting sources,” and “The system can be adapted to integrate with other surveillance”) had good rank. One indicator (“The system is not affected by fund variation”) was ranked as poor.
|The system can be adapted to accommodate new health-related events with little resources and time.||5||60||Average|
|The system can be adapted to accommodate new additional information (eg, change in case definition).||7||88||Good|
|The system is not affected by fund variation.||0||0||Poor|
|The system can be adapted to increases in the reporting sources.||7||88||Good|
|The system can be adapted to integrate with other surveillance.||8||100||Good|
The overall stability of the system scored 33%, which reveals a poor performance. Of 5 stability indicators, only one (“The system does not require time for collecting, sending, receiving, and managing data”) had a good rank (). One of the respondents said “There is no qualified coordinator capable of working continuously, and the most important obstacle is the lack of funding.”
|The system has funds from nongovernmental organizations.||0||0||Poor|
|The system has governmental funds.||1||13||Poor|
|The system is stable without sponsor funds.||0||0||Poor|
|The system does not require time for collecting, sending, receiving, and managing data.||8||100||Good|
|The reports are released regularly.||4||50||Poor|
A total of 26 stakeholders at governorate, district, and health facility levels rated the simplicity. Of 9 indicators, 5 had poor ranking. Only one indicator had good ranking. The overall simplicity of the system scored 57%, which indicates a poor performance ().
|Case definition is available.||71||55||Poor|
|Case definition is easy to apply.||95||73||Average|
|Investigation forms are available.||48||37||Poor|
|Investigation forms are easy to complete.||52||40||Poor|
|Data collection does not require telephone contact or home visit.||84||65||Average|
|Data collection does not need much time.||99||76||Average|
|Data transportation to the central level is very easy.||108||83||Good|
|Received training on NTa.||35||27||Poor|
|Data updating and follow-up of cases are easy.||77||59||Poor|
aNT: neonatal tetanus.
The acceptability indicator related to willingness to participate in NTSS had good rank. However, satisfaction with NTSS was poor. The overall acceptability score percentage was 64%, which indicates an average performance.
Timeliness and Completeness of Investigation
Of reported cases, the overall completeness of routine reporting under passive surveillance was only 23% (120/524), which indicates a poor performance. A total of 65% (340/524) of investigation forms were completed within 48 hours of the notification date, indicating an average timeliness performance, while 10% (53/524) of investigation forms were completed more than 48 hours after the notification date. The remaining (131/524, 25%) had no investigation date.
The data quality was poor, with 41% (645/1572) of the core variables missing. The highest percentage of data missing was in maternal vaccination history (96/131, 73%), followed by material applied to cord (72/131, 55%). The lowest percentage of missing was in date of birth (14/131, 11%) followed by sex variable (6/131, 5%).
Almost all (64/66, 97%) public health facilities and 26.7% (80/300) of private health facilities were covered by NTSS. The overall representativeness score percentage was only 39%, that indicates a poor representativeness.
Strengths and Weaknesses of the NTSS
The strengths of the system include easy data collection; transfer through eIDEWS; existence of surveillance staff in governorate, district, and health facility levels; several sources of reporting such as eIDEWS and CBS; and existence of new qualified leadership.
The weaknesses of the system include lack of investigation forms in the health facilities; no activities for the NTSS; lack of central supervision and communication; and no brochures, posters, or publications available on NT.
Every disease surveillance system should be analyzed and evaluated periodically to ensure it meet its objectives and improve performance wherever necessary. This evaluation revealed contradictory views about the usefulness of the NTSS, where it was generally considered poor. For example, the surveillance system did not play any role in assessing the effectiveness of the NT elimination strategy and was not capable of identifying areas at high risk. However, the surveillance data were useful to estimate NT magnitude, incidence, and mortality rates, which are the main goals for NTSS. The system might be considered useful if it satisfactorily addresses at least one of the usefulness indicators .
The NTSS demonstrated average flexibility. This result is reasonable, particularly with indicators related to the change in case definition, reporting sources, and integration with other surveillance. Receiving data from several sources such as eIDEWS and CBS may enhance the flexibility of the system. However, flexibility may be negatively affected by fund variation. Similar findings were reported by another study, which revealed its flexibility by incorporating information on measles and acute flaccid paralysis systems . In the other hand, our finding disagrees with those of a previous evaluation performed in Baluchistan, Pakistan, which demonstrated that the system had poor flexibility [ ].
Despite the ability of the system to collect, send, receive, and manage data with little resources and time, the stability of NTSS was poor. This poor stability of NTSS reflected the weak capability of the system to release reports regularly and lack of funds, especially governmental funds, which affect the system activities implementation. Therefore, the system is unstable and its sustainability is in danger.
Concerning the simplicity, the study results showed that NTSS simplicity was generally poor. This result might be due to the unavailability of case definition and investigation forms, as well as poor training, which can be attributed to lack of funds necessary for implementation of system activities. However, the transfer of the data to the central level was simple.
The findings showed that the acceptability of NTSS was average, as the majority of respondents were willing to participate in the surveillance system. Nevertheless, about two-thirds of respondents were unsatisfied with NTSS. This might be due to lack of interaction between the central and health facility levels, and this may result in lack of communication. Similarly, previous evaluation performed in Baluchistan, Pakistan, found that the system was acceptable for all stockholders with average rank .
Completeness of reporting for NT depends on conscientious notification, use of health services, health-seeking behavior, access to health services,  and a sensitive surveillance system [ , ].
Our findings revealed the completeness of investigation forms was poor. This result was far from the WHO target for completeness of investigation (≥90%) . It might be due to unavailability of investigation forms, registers, and case definitions in the health facilities as well as lack of central supervision, communication, and training courses for staff. A similar study suggested that the availability and widespread use of birth attendant logs, vital events registries, and standard case definitions in reporting sites may impact NT reporting and surveillance [ ].
The NTSS has average timeliness of investigation within 48 hours after notification. This result was lower than the WHO target for timeliness of investigation, which is ≥80% . This finding might be due to the investigation date being missing in a quarter of the investigation forms.
High-quality surveillance data and other key program indicators should be used to monitor the impact of interventions, achievement, and maintenance of NT elimination . The findings of this evaluation showed poor quality of NTSS investigation-completed forms. In addition, the percentage of missing data was high, especially data related to the maternal vaccination history and material applied to cord. In contrast, the percentage of missing data was low on the date of birth and sex variables. This might be due to lack of staff training and incentives. In comparison, a previous evaluation in Baluchistan, Pakistan, also demonstrated that the system had poor data quality [ ].
The representativeness of NTSS in general was poor in Sana’a, particularly among private health facilities. However, the representativeness of NTSS was good among public health facilities.
This evaluation has some limitations. This evaluation was limited to Sana’a due to time, budget, and security constraints. Moreover, arrival time of the investigation forms was not measured because it was not included in the investigation forms.
The overall performance of the NTSS was poor, and it is not meeting its objectives. Most of the system attributes require improvement, such as usefulness, stability, simplicity, quality of data, and completeness of investigation. In contrast, the system flexibility, acceptability, and timeliness of investigation were average. The representativeness of NTSS was poor in Sana’a, particularly among private health facilities. The most important reasons for system weakness were lack of governmental funds, program activities, staff training, central communication, and supervision. To improve the performance of NTSS, the following are recommended: capacity building of staff (focal points) by regular trainings, frequent quality checks through field visits, strengthening NTSS through technical support and government funding to ensure its sustainability, establishing electronic investigation forms for improving the system data quality, and expansion of NTSS coverage to include all private health care facilities.
The authors would like to acknowledge the Global Health Development | Eastern Mediterranean Public Health Network for their technical support.
Conflicts of Interest
Data collection questionnaire.DOCX File , 67 KB
- Hamborsky J, Kroger A. Epidemiology and Prevention of Vaccine-Preventable Diseases, E-Book: The Pink Book, 15th Edition. Washington: Public Health Foundation; 2015.
- Pan American Health Organization. Neonatal Tetanus Elimination Field Guide. Second Edition. Washington: Pan American Health Organization; 2005.
- Neonatal Tetanus Surveillance Standards. Geneva: World Health Organization; 2018. URL: https://www.who.int/immunization/monitoring_surveillance/burden/vpd/WHO_SurveillanceVaccinePreventable_14_NeonatalTetanus_R1.pdf [accessed 2019-09-15]
- Njuguna HN, Yusuf N, Raza AA, Ahmed B, Tohme RA. Elimination of Maternal and Neonatal Tetanus, 2019. MMWR Morb Mortal Wkly Rep 2020;69:515-520 [FREE Full text] [CrossRef]
- Levels and trends in child mortality 2019. UNICEF. URL: https://www.unicef.org/media/60561/file/UN-IGME-child-mortality-report-2019.pdf [accessed 2019-09-15]
- Protecting all against tetanus: guide to sustaining maternal and neonatal tetanus elimination and broadening tetanus protection for all populations. World Health Organization. 2019. URL: https://apps.who.int/iris/bitstream/handle/10665/329882/9789241515610-eng.pdf?ua=1 [accessed 2019-09-15]
- World Health Organization. Vaccine-preventable diseases: monitoring system. Global summary—Yemen. 2018. URL: http://apps.who.int/immunization_monitoring/globalsummary/countries?countrycriteria%5Bcountry%5D%5B%5D=YEM&commit=OK [accessed 2019-01-15]
- German RR, Lee LM, Horan JM, Milstein RL, Pertowski CA, Waller MN, Guidelines Working Group Centers for Disease ControlPrevention (CDC). Updated guidelines for evaluating public health surveillance systems: recommendations from the Guidelines Working Group. MMWR Recomm Rep 2001 Jul 27;50(RR-13):1-35. [Medline]
- Kakar A. Evaluation of acute flaccid paralysis (AFP) surveillance system in Balochistan. Int J Inf Dis 2014 Apr;21:239. [CrossRef]
- Mengal AA, Safdar R, Asghar J. Evaluation of Neonatal Tetanus (NNT) Surveillance System in Baluchistan, Pakistan, 2009. 2011. URL: https://www.tephinet.org/evaluation-of-neonatal-tetanus-nnt-surveillance-system-in-baluchistan%E2%80%93-pakistan-2009 [accessed 2019-01-15]
- World Health Organization. Expanded Programme on Immunization: the use of survey data to supplement disease surveillance. Wkly Epidemiol Rec 1982;57(47):361-362 [FREE Full text]
- Vandelaer J, Birmingham M, Gasse F, Kurian M, Shaw C, Garnier S. Tetanus in developing countries: an update on the Maternal and Neonatal Tetanus Elimination Initiative. Vaccine 2003 Jul 28;21(24):3442-3445. [CrossRef] [Medline]
- Lambo JA, Khahro ZH, Memon MI, Lashari MI. Completeness of reporting and case ascertainment for neonatal tetanus in rural Pakistan. Int J Infect Dis 2011 Aug;15(8):e564-e568 [FREE Full text] [CrossRef] [Medline]
|CBS: community-based surveillance|
|CDC: Centers for Disease Control and Prevention|
|eIDEWS: Electronic Disease Early Warning System|
|EMR: Eastern Mediterranean Region|
|NT: neonatal tetanus|
|NTSS: Neonatal Tetanus Surveillance System|
|WHO: World Health Organization|
Edited by M Alyahya, T Sanchez; submitted 30.01.21; peer-reviewed by R Al-Dubaiee, H Mehdizadeh; comments to author 22.02.21; revised version received 28.02.21; accepted 03.03.21; published 04.05.21Copyright
©Khaled Mohammed Al-Jamrah, Basheer Abdulgalil Al Nabehi, Khaled Abdullah Almoayed, Labiba Saeed Anam, Yousef S Khader. Originally published in JMIR Public Health and Surveillance (https://publichealth.jmir.org), 04.05.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on https://publichealth.jmir.org, as well as this copyright and license information must be included.