Added Value of Electronic Immunization Registries in Low- and Middle-Income Countries: Observational Case Study in Tanzania

Introduction

Background

With the increasing digitalization of health systems worldwide, there is growing interest and investment in electronic immunization registries (EIRs). EIRs are “confidential, computerized, population-based systems that collect and consolidate vaccination data from vaccination providers for better immunization strategies” [1]. EIRs are designed to provide a consolidated patient record to health care workers at the point of care to enable the delivery of the appropriate vaccines at the appropriate time. At the population level, EIRs can provide aggregate data on vaccination coverage to inform resource allocation and program operations. In this way, EIRs aim to improve the immunization delivery system to reach every child by supporting more effective, efficient, and data-driven care [2,3].

Vaccine coverage has historically been the primary metric for evaluating immunization programs. As an increasing number of low- and middle-income countries (LMICs) have begun implementing EIRs, vaccine coverage has been measured as a key outcome for assessing EIR effectiveness. Pre–post studies in Vietnam, Bangladesh, and Pakistan have demonstrated significant increases in child vaccination coverage after the introduction of EIRs that included SMS text message reminders and, in addition, in Pakistan, decision support systems [4-6].

In addition to improving vaccine coverage, other benefits of EIRs have been identified for individuals, immunization program performance and management, research, and population health [7,8]. For example, EIRs store patient vaccine history at the individual level, can help identify defaulters and reduce dropout rates at the program level, and provide data to support resource allocation and strategic planning at the population level [8]. Across levels, EIRs that capture individual-level data provide an opportunity to redefine traditional vaccine indicators and conduct more timely, granular analyses to support decision-making [9]. EIRs enable immunization programs to explore outcomes of interest beyond vaccine coverage, including longitudinal outcomes at the population and individual levels. As EIRs are costly to introduce and maintain, it is important for decision-makers to consider all possible benefits to justify the investment [10,11].

In some settings where EIRs are being considered or introduced, immunization coverage may already be high and, therefore, not an appropriate metric for EIR added value. The Early-Stage Digital Health Investment Tool was developed to assist ministries of health in determining their readiness to introduce a digital health tool, such as an EIR, by assessing the core building blocks of digital health [12]. In practice, country health systems with sufficient readiness are likely those that have already achieved relatively high vaccination coverage. In these contexts, improved vaccination coverage may not be the primary goal of introducing EIRs.

Objective

The aims of this study are to (1) conceptualize additional ways that EIRs can add value to immunization programs and (2) assess the feasibility and potential value-add using Tanzania as a case study.

Methods

Overview

This study comprised 2 sequential phases. First, a comprehensive list of ways EIRs can potentially add value for immunization programs was developed through stakeholder interviews. Second, the added value was evaluated using Tanzania Immunization Registry (TImR) data for a prioritized subset of program needs.

Phase 1

Conceptual Framework

A comprehensive list of common barriers that country immunization programs face in achieving coverage and equity goals was used to identify the ways in which EIRs can add value. The list was adapted from a July 2019 Gavi workshop on Improving Data use in Immunization in which approximately 40 participants from the Gavi Secretariat, core and extended partners, and country representatives identified and categorized barriers. For each common barrier, the study team (EC, TKR, and LW) identified ways that EIRs could help address the barrier based on their expertise and implementation experience.

Data Collection and Analysis

Stakeholder interviews were conducted to refine the framework of the immunization program barriers and potential EIR solutions. Stakeholders were purposively selected based on their expertise in research, policy, or implementation of EIRs. A total of 7 stakeholders participated in semistructured web-based interviews facilitated by the study team (EC) from November 2019 to January 2020. A total of 4 stakeholders were government officials from countries in Sub-Saharan Africa, identified through the BID (Better Immunization Data) Learning Network [13]. A total of 3 stakeholders were from international public health agencies, donors, or implementing organizations. Summary notes from the interviews were used to refine the conceptual framework. A follow-up web-based survey (using SurveyMonkey, Momentive, Inc) was sent to a wider group of EIR experts (including interviewees) in January 2020, asking respondents to prioritize topics from the conceptual framework for further analyses. Survey responses from 17 individuals were used, in conjunction with the interview data, to prioritize the 4 topics for phase 2 analyses.

Phase 2

Setting

Data from Tanzania’s EIR were analyzed to illustrate how an EIR can add value to each of the prioritized topic areas. The Government of Tanzania partnered with the BID Initiative, funded by the Bill & Melinda Gates Foundation and launched in 2013, to design and implement a package of solutions to improve immunization data quality and use [14]. An EIR was an essential component of the solution package. EIR design began in 2014 and went through iterations culminating in TImR, which is built on the OpenIZ platform (now known as SanteDB, SanteSuite) [15]. The Government of Tanzania has led a staged rollout of TImR to facilities across districts and regions. As of December 31, 2020, TImR was rolled out to 3736 facilities across 15 of 25 regions in mainland Tanzania and included 1.6 million client records.

Data Sources

Immunization, facility, and patient data were extracted from the TImR system with permission from the Government of Tanzania. Data were deidentified after extraction, and all analyses were conducted using deidentified data. The development and implementation of the TImR system have been discussed in detail elsewhere [15-17]. Population density data were extracted at the ward level from WorldPop’s United Nations–adjusted GeoTIFFs at 100×100 km spatial resolution using Database of Global Administrative Areas (GADM) administrative shapefiles and matched to facilities based on facility geocodes [18,19]. Subject matter experts were consulted on the construction of analysis variables (eg, missed opportunities for vaccination [MOVs]). Data were processed and analyzed using Alteryx (version 2020.3; Alteryx, Inc), R (version 4.0.0; R Foundation for Statistical Computing), Tableau (version 2020.2; Tableau Software, Inc), and STATA (version 14.2; StataCorp LLC).

Ethics Approval

This study received nonresearch determination from the PATH. The Government of Tanzania and the PATH have data-sharing permissions in place that guided the use of TImR data for this study.

Data Restrictions

The analyses focused on services provided between 2017 and 2019 and the vaccine doses that were included in the official Tanzania vaccine schedule, specifically Bacillus Calmette–Guérin (BCG); oral polio vaccine (OPV); diphtheria, tetanus, pertussis, hepatitis B, and Haemophilus influenzae type b (Penta); pneumococcal conjugate vaccine (PCV); rotavirus (Rota); and measles-containing vaccine (Table 1). Data were further restricted to doses received while the TImR system was live (ie, doses logged in the system at the time of service or shortly afterward). Doses retroactively entered to complete patient medical records were included in the continuum of care analysis only. This analysis included back-entered doses for patients with at least one live TImR entry to capture the full picture of their vaccine history. A total of 3 regions (Mtwara, Rukwa, and Ruvuma) with <50 visits recorded in TImR by December 31, 2019, were excluded. In addition, patient IDs with >3 instances of a given vaccine dose (eg, OPV-1) were assumed to be dummy patient IDs used to log vaccinations provided during mass immunization campaigns and were excluded from the analysis. Patient IDs with up to 3 instances of a given dose were assumed to result from data entry errors.

Definitions

Outcomes

Missed Opportunities for Vaccination

MOVs were assessed at the visit level using the World Health Organization (WHO) definition: “any contact with health services by an individual (child or person of any age) who is eligible for vaccination (e.g., unvaccinated or partially vaccinated and free of contraindications to vaccination), which does not result in the person receiving one or more of the vaccine doses for which he or she is eligible” [20]. Dose eligibility was based on patient age, prior doses received, and time since the last dose of the vaccine sequence (if applicable). The MOV variable was constructed both as a binary (any MOV in a visit or not) and count (number of vaccine-specific MOVs per visit). Binary coding was used for the regression models, which motivated the use of logistic regression.

Vaccine Dropout

Vaccine-specific dropout for multidose vaccines was defined as receiving the first but not the last dose in the vaccine schedule (eg, receiving PCV-1 but not PCV-3). OPV dropout was defined as receiving OPV-0 or OPV-1 and not OPV-3; children who did not receive OPV-0 by the age of 2 were eligible for OPV-1 without receiving OPV-0; therefore, either vaccine can be treated as the starting dose. We also assessed dropout between birth doses and first follow-up visit, defined as receiving either of the birth doses (BCG or OPV-0) but none of the first follow-up visit doses (OPV-1, Penta-1, PCV-1, and Rota-1). Finally, we assessed overall dropout, which is defined as receiving at least one scheduled vaccine dose but not completing the full 14-dose schedule. The dropout variables were constructed as binaries (meeting criteria for dropout or not), motivating the use of logistic regression in the models.

Assigned Facility and Nonassigned Visits

Children are assigned a home facility when they are registered in the TImR system based on their preferences and where they plan to receive care. A nonassigned visit is a visit to any health facility other than the assigned visit. This variable was constructed as a binary variable (visit at home facility or not), motivating the use of logistic regression in the models.

Predictors

Dose Timeliness

A dose was considered timely if it was received within 7 days of the scheduled date (Table 1). In practice, in Tanzania, a child is generally considered a defaulter after 7 days past their scheduled date.

Urbanicity

A facility was designated as urban if the ward in which it is located had a population density of at least 500 persons per square km and rural if otherwise [21]. A patient was assumed to live in an urban area if their assigned facility, presumably near their residence, was designated as urban or rural, if otherwise.

Stockout

Facility stock use, including days of 0 stock, is recorded in TImR by facility and vaccine type. Vaccine-specific stockout was defined as any period in which the stock balance for a given vaccine was zero. A composite indicator was also constructed for the proportion of days with a stockout, with the number of days with a stockout for a primary vaccine (BCG, OPV, PCV, Penta, Rota, or measles-containing vaccine [MCV]) as the numerator and the number of days with facility stock data in the TImR system for each primary vaccine as the denominator.

Age

Age was defined in two ways: static age at the time of data extraction (December 31, 2019) and age at the time of a given visit. The 2 age variables were coded into 1-year categories up to the age of 5 years (ie, <12 months, 12-23 months, 24-35 months, 36-47 months, and 48-59 months), which is the upper limit of standard eligibility for most of the vaccines of interest.

Regression Models

For all analyses, we used mixed-effects logistic regression to assess the factors associated with the various outcomes. In all models, relevant patient and facility characteristics were included as fixed effects, and nested random intercepts for region, district, and facility ID were used to account for clustering.

Results

Phase 1: EIR Added Value

Textbox 1 lists ways that EIRs can help address common barriers faced by immunization programs in achieving coverage and equity. EIRs can add value through existing functionalities (eg, the ability to identify underimmunized children) and through functionalities that may not be a core component of existing systems (eg, the ability to serve as a platform for remote, virtual supportive supervision).

On the basis of stakeholder input, 4 topics were prioritized for phase 2 analyses:

1. Denominators and population movements, including patient movement between facilities or geographic areas for care
2. MOVs, including their frequency and any associated characteristics
3. Continuum of care, including which children drop out and when in the vaccination schedule
4. Continuous quality improvement (CQI), including trends or outliers in data quality or service delivery, to inform targeted quality improvement efforts

The remainder of this section provides an overview of the TImR data and then provides results on each of the 4 priority topic areas to illustrate how EIR data can be used to better understand denominators and population movement, MOVs, continuum of care, and CQI.

Phase 2: Tanzania Case Study

The sample size for the individual analyses varied because of differing inclusion criteria and missing data. In full, our sample comprised 2,444,803 vaccine doses over 958,870 visits for 559,542 patients. These visits occurred in 2359 health facilities covering 57 districts in 10 regions. The median (IQR) number of provided doses per facility per month was 40 (9-123), and the median number of visits was 17 (4-49). Table 2 provides participant demographics and facility characteristics.

Denominators and Population Movement

Overview

This analysis explored population movement, that is, care seeking at alternative (nonassigned) facilities, which affects the accuracy of facility denominators. Of 810,568 total visits, 765,835 (94.48%) were at a child’s assigned facility, 15,575 (1.92%) were at a nonassigned facility within 5 km of the child’s assigned facility, 14,147 (1.82%) at facilities located >5 km from the assigned facility but within the same district, 12,267 (1.51%) in a different district within the same region, and 2926 (0.36%) in a different region. Figure 1 summarizes attendance by region for all visits and visits to nonassigned facilities. Although children were similarly likely to seek care at their assigned facility across regions, patterns of care seeking to nonassigned facilities varied. For example, of visits to nonassigned facilities, children in Dar Es Salaam region were most likely to seek care within 5 km (64/85, 75%), whereas children in Geita were most likely to seek care outside of the region (131/187, 70.1%).

Table 3 explores visits at assigned and nonassigned facilities based on patient and assigned facility characteristics. There was little variation in the likelihood of a visit being at a nonassigned facility based on patient sex, assigned facility ownership, or assigned facility type. As expected, patients assigned to urban facilities and patients whose assigned facility had a higher number of facilities within 5 km were more likely to visit nonassigned facilities. Older children were more likely to visit nonassigned facilities; however, this could be an artifact of older children having more visits (and thus more opportunities to visit other facilities) or being more likely to have moved since being entered into the TImR system (ie, no longer residing near their assigned facility). Patients assigned to facilities newer to the TImR system were less likely to visit nonassigned facilities.

‎View this figure

Spatial Variation in Assigned Facility Attendance

Figure 2 shows the proportion of all visits by children assigned to a given facility that occurred at the assigned facility. Facilities with low attendance appeared to cluster in northern Kilimanjaro, southeastern Arusha, southeastern and urban Mwanza, and coastal and central Tanga.

‎View this figure

Model Results

Table 4 shows results from the logistic regression model with a given visit to a nonassigned facility as the outcome of interest. Children assigned to public facilities and health centers or hospitals, facilities with a longer duration of TImR use, and facilities in areas with higher health facility density were significantly more likely to visit a nonassigned facility. Children attending facilities with a greater number of recorded visits were significantly less likely to visit a nonassigned facility. Interestingly, the relationship with age was no longer monotonic after adjusting for other covariates. As compared with children aged ≤12 months, children aged 1 to 2 years were less likely to visit a nonassigned facility, whereas those aged 3 to 4 years were more likely.

Missed Opportunities for Vaccination

Overview

MOVs, where the patient did not receive at least one vaccine for which they were eligible, were observed in 23.69% (226,525/956,195) of visits. Although we found little variation in the likelihood of an MOV based on sex, there was notable heterogeneity across age groups, facility urbanicity, facility type, and duration of TImR use at the facility (Table 5). The higher likelihood of an MOV among younger patients may be an artifact of the higher number of scheduled doses in the first year of life, and therefore, greater opportunity for missed doses.

Of the 557,674 children included in the analysis, 167,115 (29.97%) had ≥1 MOVs. The mean number of MOVs per child was 0.61 (SD 1.20). Among the 167,115 children with an MOV, 85,697 (51.28%) had ≥1 MOV (range 1-15). Of 338,439 recorded MOVs, rotavirus was the most likely to have an MOV (accounting for 28.26% of all MOVs; n=95,650), followed by PCV (18.82%, 63,682), Penta (17.84%, 60,363), OPV (17.15%, 58,039), BCG (16.23%, 54,924), and MCV (1.71%, 5781). The lower MOV proportion for MCV was likely because of fewer visits where children were age-eligible for MCV (aged at least 9 months).

MOV Reasons

The TImR system allows providers to indicate the reasons why a scheduled and eligible dose was not provided. However, the reason will only be noted if a dose is knowingly not given and thus is absent for doses for which providers did not recognize the patient’s eligibility. For eligible doses that the provider logged as missed, the data indicated the mechanisms behind MOVs.

Table 6 details MOV reasons by vaccine type. Of 338,439 recorded MOVs, 183,623 (54.26%) had a listed reason: 177,624 (52.48%) were because of facility stockout, 2474 (0.73%) because of medical contraindication, 3184 (0.94%) because of being late (generally meant to indicate that the child is too old to start the vaccine sequence), 178 (0.05%) because of guardian refusal, and 163 (0.05%) because of expired stock. These reasons varied by vaccine type, with roughly three-quarters of Penta and PCV MOVs because of stockout but less than half for BCG, MCV, and rotavirus MOVs. Rotavirus MOVs were more likely to result from medical contraindications (913/95,650, 0.95%) compared with MOVs of the other vaccine types, whereas MCV had the highest likelihood of being missed because of guardian refusal (15/5781, 0.26%).

Model Results

Results from the any-vaccine MOV and OPV-specific MOV models were selected as illustrative examples of interest and are shown in Table 7. Unadjusted results can be found in Multimedia Appendices 1 and 2. Age group and TImR duration were significantly associated with any MOV and OPV-specific MOVs. Compared with children aged <1 year, older children were substantially less likely to experience MOVs in both models. This may be because of the greater opportunity for MOVs at younger ages because of more scheduled doses in the first year of life. Interestingly, TImR use duration at the time of visit showed opposite directionality between the models, with longer TImR implementation associated with a higher likelihood of any MOV but lower likelihood of OPV-specific MOVs, suggesting that there may be different mechanisms leading to MOVs by vaccine type.

Continuum of Care

This analysis explored the vaccine dropout. To ensure common eligibility for doses, this analysis was restricted to children aged 12 to 47 months at the end of 2019 and focused on the 14 doses scheduled for the first year of life (ie, through MCV-1; Table 1).

Immunization Coverage

Overall, 93,619 (31.79%) of 294,464 children in our sample were fully immunized for doses scheduled in the first year of life (inclusive of OPV-0), with a further 39,718 (13.48%) receiving all scheduled doses, except for MCV-1. Figure 3 shows the doses received and timeliness by vaccine type and dose. As expected, there was a drop-off in coverage with later doses in each vaccine sequence. Timeliness also decreased monotonically with later doses in a sequence.

‎View this figure

Vaccine Dropout

Table 8 details dropout rates across patient characteristics. For multidose vaccine-specific dropout, OPV had the highest rate (66,798/217796, 30.67%), followed by PCV (78,767/268,582, 29.33%), Penta (76,659/268,315, 28.57%), and Rota (52,086/255,337, 20.4%). Rotavirus may have had a lower dropout rate because there were only 2 doses in the sequence. There were common trends for all outcomes, such as older children and private facilities showing lower levels of dropout for all types of dropouts. However, some trends were outcome-/vaccine-specific, such as rural facilities showing higher levels of dropout for all vaccines except OPV. Most of these differences were marginal except for dropout by age group.

Immunization Typologies

To better understand vaccination profiles, we constructed immunization archetypes using all possible combinations of eligible scheduled doses. Patients were fit into these archetypes based on their immunization history. Table 9 shows the 10 most common archetypes by vaccine doses received. After being fully vaccinated and fully vaccinated except for MCV-1, the third most common typology was receiving all doses except for the OPV sequence (19,322/294,464, 6.56% of children), followed by dropping out between the second and third visits (13,270/294,464, 4.51%) or between the third and fourth visits (13,102/294,464, 4.45%), and receiving only the birth doses (BCG and OPV-0; 10,156/294,464, 3.45%).

Visit Dropout

To understand dropout between different scheduled visits, we analyzed the proportion of children that had received any vaccine from each of the 5 scheduled touchpoints with the immunization system in the first year of life (Table 1). Overall, 96.29% (283,548/294,464) of children received at least one of the birth doses (BCG or OPV-0), 93.16% (274,314/294,464) received at least one of the visit 2 doses (OPV-1, PCV-1, Penta-1, or Rota-1), 80.98% (238,450/294,464) received at least one visit 3 dose, 67.52% (198,812/294,464) received at least one visit 4 dose, and 48.21% (141,948/294,464) received the visit 5 dose.

Model Results

The results from the Penta and overall dropout models were selected as illustrative examples of interest and are shown below. Unadjusted results can be found in Multimedia Appendices 3 and 4. As shown in Table 10, older age was significantly associated with a lower likelihood of both Penta and overall dropout (ie, starting but not finishing the 14-dose schedule). In the overall dropout model, urban facilities were associated with a significantly lower likelihood of overall dropout, and public facilities were associated with a higher likelihood. These trends were not observed in the Penta dropout model.

Continuous Quality Improvement

EIRs provide data for the rapid assessment of CQI improvement measures. These assessments can help improve service provision by identifying areas in need of targeted training or other quality improvement interventions. As shown in Table 11, 10% of facilities account for most of the issues, suggesting that targeted interventions to identified facilities could greatly improve care. These results use absolute numbers and, therefore, will be biased toward facilities with higher patient loads and longer TImR implementation durations. In practice, CQI analyses would likely be restricted to specific months or quarters, reducing any duration bias. Absolute figures can also offer greater efficiency by targeting CQI interventions to providers or facilities with the highest absolute number of issues.

Discussion

Principal Findings

EIRs can add value in multiple ways. Access to individual-level data that captures all touchpoints with the immunization program allows for new analyses that can benefit immunization programs, national and regional ministry staff, health care providers and administrators, funders, and other stakeholders [8,9,22]. Descriptive statistics can be used to rapidly monitor service provision and vaccination coverage or inform quality improvement efforts. Longitudinal and spatial analyses can be used to understand temporospatial changes in care and coverage. Risk factor analyses can be used to identify patient and facility characteristics associated with immunization issues (eg, dropout). These analyses can be targeted to the relevant stakeholder groups. For example, facility-level statistics for a given district can inform targeted supportive supervision, and national-level coverage trends can enable evidence-based policy development. EIRs also allow for more cost-effective and rapid synthesis of immunization data; many of these descriptive statistics and analyses would not be possible using the aggregate data available in the routine health information system or would require significant additional funding, time, and other resources for survey data collection [9]. The analyses presented in this study are intended to illustrate the types of insights that EIR data can provide to immunization programs.

Denominators and Population Movement

Inaccurate population denominators are a common challenge for monitoring coverage, improving implementation, and informing planning, such as projecting vaccine stock and staffing needs. A recent scoping review of immunization data quality in LMICs found that denominators were often inaccurate, infrequently adjusted, and inconsistent between the district and national levels [23]. Population denominators are influenced by migration, urbanization, and refugee crises, among other population dynamics that can have large effects at the local level [24]. Population denominators are further complicated by children seeking care at different facilities over time. The Strategic Advisory Group of Experts working group on the quality and use of global immunization and surveillance data identified inaccurate denominators as a common challenge and noted the lack of guidance on how to improve the accuracy of denominators and track mobile populations [25].

EIRs greatly simplify tracking patients who seek care at multiple facilities, enabling a more nuanced understanding of population movement across both geography and time and allowing for more robust coverage estimates. The use of the TImR data allowed us to explore both the magnitude of and factors associated with seeking care at facilities other than the patient’s assigned, or home, facility. Our analysis revealed that a small subset of patients sought care outside their assigned facility (44,733/810,568, 5.52%); however, this varied by region; facility urbanicity, type, ownership, patient volume, and duration of TImR system use; density of facilities in the immediate area; and patient age. In addition, where patients seeking care varied by region, patients in some regions were more likely to travel to other districts and regions for care. For example, children who did not attend their assigned facility in Geita were most likely to attend a facility outside of the region, potentially because of population mobility associated with mining in the region. These insights can help inform resource allocation. EIRs also greatly simplify tracking patients who seek care at multiple facilities, decreasing the likelihood of missed or redundant doses. Although a small number of children in the Tanzania case study sought care outside their assigned facilities, some areas would have a much larger nomadic or mobile population. For example, in Cameroon, children born at home, immigrants, emigrants, and nomadic populations are not accurately accounted for when planning outreach vaccination sessions, which contributes to delaying or not vaccinating an estimated 30% to 70% of the population in some districts [26].

Missed Opportunities for Vaccination

Identifying and avoiding MOVs is an important and cost-effective method for achieving greater vaccination coverage. The challenge is in identifying when, where, and among which children or facilities MOVs are experienced to address them. Integration of clinical decision support systems within EIRs can automate the determination of child dose eligibility and alert the provider, which has been shown to reduce MOVs for routine childhood immunizations [27,28]. In addition, by collating vaccination history with child and facility characteristics, EIRs naturally allow for exploration of MOVs across these characteristics and by different vaccines. Our analysis of the TImR data showed that MOVs were highest among children aged <12 months (as mentioned, potentially because of the higher number of scheduled doses in the first year of life); however, there were few significant differences by other individual or facility characteristics. Other studies of countries in Sub-Saharan Africa have identified additional demographic and socioeconomic characteristics associated with increased odds of MOVs, including high birth order, high number of under 5 children in the house, lack of maternal education, lack of media access, and household and neighborhood poverty [29,30]. Although these data were not captured in TImR, they could be captured by an EIR to enable new analyses and equity insights [8,31].

This information can be used by providers to identify children who may be at higher risk of experiencing an MOV. In addition, it can be used by managers to identify providers and facilities with higher rates of MOVs for supportive supervision or refresher training or identify areas with high rates of vaccine hesitancy for outreach campaigns. In addition, EIRs can provide insight into the mechanisms behind MOVs, such as stock issues and vaccine-specific hesitancy. Where data were available, stockouts were the primary reason for MOVs, whereas mechanisms such as vaccine hesitancy and medical contraindications were relatively rare. The TImR data also showed that rotavirus was the most likely to have an MOV, which may indicate that eligibility requirements should be reviewed or refresher training provided. For additional insights, analysis of EIR data can be complemented by other tools such as those included in the WHO MOV strategy toolkit [20].

Continuum of Care

Identifying where in the vaccine schedule some children drop out and why they drop out is another key challenge for achieving high levels of vaccine coverage. Understanding which vaccine doses and child and facility characteristics are associated with failure to complete a vaccine sequence or the full vaccine schedule can help inform service provision, training, and quality improvement measures at the facility, regional, and national levels. In the TImR data, nearly half of children aged 12 to 35 months were fully vaccinated or had received all doses except MCV-1 of the 14-dose under-12-month schedule (ie, through MCV-1). Among children who did not complete the vaccine schedule, levels of dropout varied by vaccine. Facility characteristics associated with dropout also varied by vaccine; for example, assigned facility urbanicity was significantly associated with a lower likelihood of overall dropout (ie, starting but not finishing the 14-dose schedule) but not Penta-specific dropout, suggesting that the mechanisms behind dropout may vary by vaccine. Continuum of care analyses could be further expanded if the EIR data were linked to a birth registration system. In the Tanzania case study, 5.79% (16,414/283,548) of children dropped out between birth and the first immunization dose; however, this may be an underestimation if some children are not registered at birth. In countries with a strong civil registration and vital statistics system, linking the EIR to birth registration or an antenatal care registry could expand the continuum of care analysis. Using EIRs to explore immunization typologies can also provide insight into which vaccines and visits require greater care. For example, in the TImR data, 6.56% (19,322/294,464) of children were fully vaccinated through MCV-1 except for the 3 to 4 OPV doses, highlighting the need for greater research into barriers to OPV coverage.

CQI Analysis

The CQI analysis showed that most issues (eg, MOVs) came from a minority of facilities. EIRs enable decision-makers at the national and subnational levels to quickly assess and identify providers, facilities, or geographic areas for targeted quality improvement measures, thereby improving the quality of care and increasing improvement in intervention effectiveness.

Added Value of EIRs

These analyses were designed to show the potential of EIRs to allow for a more nuanced, rapid, and cost-effective evaluation of vaccine program data to facilitate data use for decision-making. For example, automated dashboards of key indicators (eg, vaccine-specific coverage, stockouts, and child dropout) can inform planning and clinical practice at the facility level without the need for on-site data analysis. Providers can also use EIRs to simplify the tracking of individual patients, particularly those seeking care at multiple facilities, to improve the quality of care and reduce issues such as MOVs [27,28]. The integration of EIRs with SMS text messaging services to automate appointment and outreach to children at risk of defaulting has been shown to reduce dropout rates for routine childhood immunizations [4]. At the district and regional levels, the evaluation of underperforming facilities can be used for targeted supportive supervision and supplemental training. At the national level, up-to-date data on geographic and spatial trends in vaccine coverage can be used to inform nationwide campaigns, resource allocation, or policy development.

Designed well, EIRs can democratize immunization data. However, they require the necessary support to function effectively. The Early-Stage Digital Health Investment Tool has identified 6 building blocks for effective digital health systems: human capacity, investments and funding, data capture and use, infrastructure, standards and interoperability, and governance and policy [12]. Strong building blocks can maximize the effectiveness of EIR systems; however, this can pose a challenge in some low- and middle-income settings where 1 or several of these building blocks may be lacking. The WHO estimates that 50% of low-income and 24% of lower-middle–income countries have strong institutional capacity or involvement in data analysis at the national ministry [22]. With technical capacity often centralized at the national level, these figures are even lower at the subnational level. Furthermore, 54% of low-income and 41% of lower-middle–income countries are rated nascent, limited, or moderate in their capacity to have data and evidence drive policy and planning [22]. Implementing robust and routinized data frameworks, including EIRs, can address gaps in data availability and provide mechanisms to harness the data to drive evidence-based policy and planning. Automation and tailoring of data output to specific end users (eg, facility-level indicator dashboards for providers) can simplify data analysis and interpretation. However, effective use of EIR data for decision-making will require health care workers and administrators at all levels to have the skills, motivation, and autonomy to understand and act on the data [16,17]. Leadership at the national and regional levels should prioritize capacity building to enable the health system to make use of EIR data [32].

EIR is a solution that aims to improve immunization program performance. The efficiency and impact of EIRs can be maximized by introducing them in combination with other interventions, such as capacity strengthening for data use, vaccine stock management systems, data governance frameworks, or SMS text messaging reminders for caregivers. Interventions that use multiple mechanisms to address various barriers to data use have been found to be more successful in achieving immunization data use and action [33].

Limitations

The TImR results are intended to illustrate the ways EIRs can add value to immunization programs by providing actionable information for health care providers and managers. The results are not intended to be generalizable to Tanzania as a whole because of several data limitations. First, regions and districts implemented TImR at various points in time, meaning that some geographies are over- or underrepresented in the results. Second, and relatedly, only a subset of regions in Tanzania have introduced TImR; therefore, immunization services delivered outside the TImR coverage area are not captured in the results. Third, children who may live within the TImR coverage area but have not had a touchpoint with the immunization delivery system (also known as zero dose children) or were not registered at birth are not captured in the results. Fourth, this study did not assess data completeness, and any incomplete data (eg, providers not entering all immunizations into TImR) may limit the accuracy of the results. Fifth, prior studies of the TImR data have shown reduced system use over time, potentially biasing results toward facilities with greater capability to maintain reporting systems [16]. Sixth, as mentioned earlier, the analyses were limited to data captured in TImR. Although these data can be powerful for diagnosing issues, they do not capture all patient, facility, or geographic characteristics that may influence immunization delivery and can be limited in explaining trends. For example, MOVs may be underestimated as this study only captures MOVs during immunization visits and not during nonimmunization visits [34,35]. As noted earlier, some characteristics shown to be associated with MOVs were not captured in TImR. Triangulation with other data sources or targeted follow-up data collection can help answer the why questions. Finally, the lens used in this study was the assessment of the potential added value of EIRs. This study does not attempt to highlight the challenges associated with implementing or maintaining EIRs, although many such challenges have been identified elsewhere [1,3,8,9,17,36].

Conclusions

EIRs have the potential to add substantial value to immunization stakeholders at all levels of the health system beyond measuring vaccine coverage. Individual-level data captured through EIRs can enable new analyses to understand immunization service delivery or care-seeking patterns, potential risk factors for underimmunization, and where challenges occur. Notably, most issues (eg, occurrence of MOVs, visits to a nonassigned facility, and number of defaulters) occur in a minority of facilities, highlighting the potential for EIRs to inform targeted quality improvement efforts. However, to achieve this potential, country programs need to leverage and strengthen their capacity for collecting, analyzing, and interpreting the data. Measures and analyses should be prioritized to match the needs and capabilities of the immunization program. Ideally, the prioritized measures should be integrated into routine systems to facilitate ongoing CQI efforts. As EIRs are introduced and scaled in LMICs, implementers and researchers should continue to share real-world examples and build an evidence base for how EIRs can add value to immunization programs, particularly for innovative uses.