Clinical laboratories provide critically important data for disease investigations and supply timely information for public health. Laboratory results help identify and characterize pathogens, conduct routine surveillance, and respond to communicable and noncommunicable diseases. Effective and timely laboratory services help achieve national disease priorities and global targets including the Sustainable Development Goals and the 95-95-95 targets launched by the Joint United Nations Programme on HIV/AIDS [1-5]. Clusters of differentiation 4 (CD4) testing is critical for monitoring the immune status of people living with HIV and the risk of opportunistic infections. Health care providers depend on laboratory results to decide on differentiated care models and tuberculosis screening [6]. For a national HIV program to be highly successful and reduce HIV-related deaths, the health system must integrate data from an effective clinical laboratory system. Access to quality laboratory services remains a challenge in low- and middle-income countries (LMICs), and many still rely on paper-based records [7]. Throughout the cascade of care, paper documents and records require a careful chain of possession, tracking, and version control, and their use and sometimes loss can impede an already complex process of service delivery. Problems in data quality could occur at every step of this manual process, jeopardizing the quality of care.

Laboratory information systems (LISs) are an important part of the infrastructure for laboratory operation and information management. According to the requirements for quality and competence of medical laboratories issued by the International Organization for Standardization (ISO), most of the information management requirements are for electronic systems [8]. This signals that, to reach ISO standards, laboratories should use electronic rather than paper-based LISs. The World Health Organization established the Stepwise Laboratory Quality Improvement Process Toward Accreditation (SLIPTA) checklist to guide clinical laboratories in Africa through continuous quality improvement toward accreditation to international standards [9]. The SLIPTA checklist highlights computerized LISs as an integral step to ensuring laboratory operation and fulfilling the criteria of laboratory information management on ensuring data quality, data storage and backup, and patient confidentiality [9]. The lack of effective electronic LISs is one of the barriers to quality laboratory service delivery in LMICs [10].

LISs can also improve the quality of care, patient safety, and disease surveillance [10-14]. Compared to paper-based information systems, electronic LISs provide more timely and more accurate monitoring and reporting of turnaround time, test failure rates, and other laboratory quality indicators essential to informing clinical care [10]. The features in the Open-Source Enterprise-Level Laboratory Information System (OpenELIS) are intended to ensure data timeliness, completeness, and validity. Control elements and graphical widgets such as drop-down menus and radio buttons make data entry easier and faster, supporting data timeliness and validity. Required data fields must be completed to generate the laboratory result for a patient, assuring data completeness. Valid ranges displayed alongside each test result, the flags for manual review of out-of-range results, and the mandatory validation step before generating results help ensure data validity.

The International Training and Education Center for Health (I-TECH) has been collaborating with the Ministry of Health, Public Hygiene, and Universal Health Coverage (MSHPCMU) in Côte d'Ivoire since 2009 with funding from the United States President’s Emergency Plan for AIDS Relief to build a functional, accredited diagnostic and laboratory network for effective HIV care and treatment. One of the key program interventions is establishing and supporting the use of OpenELIS interfacing with laboratory testing analyzers [15]. From 2009 to 2021, OpenELIS was installed at 108 laboratories in Côte d'Ivoire, including 68 sites that use it primarily for HIV data, 1 for food and drug safety data, 27 for tuberculosis data only, and 12 sites that use OpenELIS for routine laboratory testing data.

This study aims to estimate the effect of OpenELIS on data quality indicators including data completeness, timeliness, and validity, using CD4 testing data. Specifically, we aimed to answer the following questions: (1) Did the use of OpenELIS in Côte d’Ivoire’s clinical laboratories lead to statistically significant improvements in the data quality outcomes? (2) If so, what was the magnitude of the improvements compared to the counterfactual scenario in which the laboratories had not adopted OpenELIS? and (3) Had the improvements persisted after the initial adoption of OpenELIS?

Beyond a single study of the effect of an LIS on laboratory turnaround time in Peru [16], there is limited evidence on how LIS interventions enhance data quality in LMICs. Findings from our study will provide evidence on the effectiveness of the OpenELIS intervention in improving data quality; inform policy recommendations and guidance for the Côte d'Ivoire MSHPCMU; and inform decision makers elsewhere to consider adopting OpenELIS in countries with a context similar to Côte d'Ivoire.

The number of laboratory weeks in the analyses were 613, 1820, and 1629 for timeliness, completeness, and validity, respectively. The analysis included an average of 34 weeks of preimplementation data and 53 weeks of postimplementation data from each of the 21 laboratories. A total of 24,381 and 40,040 HIV client records from the pre- and postimplementation periods, respectively, were in the analysis (Table 1).

Only 3 of the 21 laboratories had all 48 weeks of complete data on sample reception before the laboratories adopted OpenELIS and when they used paper registries exclusively. A total of 10 laboratories had all 48 weeks of complete data on test results. After adopting OpenELIS, 19 of the 21 laboratories had 72 weeks of complete data for both sample reception and test results.

Before adopting OpenELIS, a typical laboratory had minor improvements in the 3 data quality outcomes that were not statistically significant (timeliness: OR 1.03, 95% CI 0.99-1.07; P=.12; completeness: OR 1.11, 95% CI 0.93-1.32; P=.24; and validity: OR 1.00, 95% CI 0.99-1.02; P=.76; Table 2). Within the first week of OpenELIS implementation, we observed an immediate level change in data timeliness, completeness, and validity (Figure 3). After adopting OpenELIS, the laboratories had 5.27-fold greater odds of timely results (OR 5.27, 95% CI 4.33-6.41; P<.001); 3.59-fold greater odds of complete data for the required fields (OR 3.59, 95% CI 2.40-5.37; P<.001); and 1.34-fold greater odds of valid results (OR 1.34, 95% CI 0.69-2.60; P=.38; Table 2). The immediate improvements in timeliness and completeness were both programmatically substantial and statistically significant.

Starting from the second week of OpenELIS implementation, we observed a slope change in the trend of completeness. The average odds of test results having complete data for the required fields each week thereafter were 1.03 times the odds had the laboratories not started using OpenELIS (OR 1.03, 95% CI 1.02-1.05; P<.001). In fact, the average proportion of test results having complete data stayed at almost 100% after a laboratory started using OpenELIS, while the average proportion declined over time in the counterfactual had there been no OpenELIS (Figure 3B). There was no significant change in the trend of timeliness and validity (timeliness: OR 1.01, 95% CI 0.99-1.02; P=.49 and validity: OR 0.99, 95% CI 0.97-1.01; P=.55).

In terms of heterogeneity at the laboratory level, laboratories at local general hospitals seemed to have more substantial improvements in timeliness immediately after starting to use OpenELIS and have maintained the improvements better over time as compared to regional laboratories (Figure S1 in Multimedia Appendix 1). Since a greater number of regional laboratories had less complete data than laboratories at general hospitals before OpenELIS implementation, immediate improvements in completeness were more obvious at regional laboratories, and the improvements also maintained over time (Figure S2 in Multimedia Appendix 1). Consistent with the result at the aggregate level, no laboratories had changes in validity (Figure S3 in Multimedia Appendix 1).

This study shows that data availability improved after clinical laboratories in Côte d’Ivoire adopted OpenELIS, and there was rapid and substantial improvement in data timeliness and completeness. Following the immediate post-OpenELIS implementation improvements, both timeliness and completeness remained close to 100%. Data validity was very close to 100% both before and after OpenELIS adoption. There was some heterogeneity in the results at the laboratory level, with laboratories at local general hospitals showing more substantial improvements in timeliness while regional laboratories showed more obvious immediate improvements in completeness.

To our knowledge, this is the first study to demonstrate the immediate and maintained effect of a scaled, multisite LIS on data quality in sub-Saharan Africa. There are limited studies from LMICs despite significant financial investments from the international donor community. Previous studies from LMICs took place on other continents or did not include as many laboratories or subnational geographic areas as ours did. The e-Chasqui LIS for tuberculosis testing data at 14 laboratories in Peru was the only one that was implemented and evaluated through a cluster randomized controlled trial [21]. The trial showed that the LIS decreased turnaround time and improved quality of care, but the study did not examine the effect on data quality [21]. Our study contributes to the evidence by quantifying the effect size of data quality improvement and its change over a period of 30 months at 21 clinical laboratories. The findings offer critical information for policy makers and digital health researchers in LMICs when considering the real-world effectiveness of the OpenELIS software or digitalization of a sub-Saharan African laboratory system.

Systematic implementation of a single LIS across an entire national health system is rare in LMICs. A rapid literature review of peer-reviewed journal articles identified 8 different LISs that 6 countries have implemented at more than 1 site nationally, namely Ethiopia, Malawi, Namibia, South Africa, Peru, and Vietnam [16,21-30]. The LIS in South Africa is the only 1 that has the capacity to record data on all laboratory testing services at all clinical laboratories in the country [22]. It is likely that, as a result of the nationwide LIS adoption in South Africa, usage of laboratory data in research has also increased, since almost 20% of the rapid literature review search results from LMICs were epidemiological studies from South Africa using data from the LIS [31-39].

Part of the success of OpenELIS in Côte d’Ivoire can be attributed to the systematic approach taken by I-TECH in establishing and supporting the use of OpenELIS. The approach included the endorsement by the MSHPCMU and locally trained technical staff who facilitated implementation and provided continuous technical support of the system to the laboratories and their users. The relative advantages of OpenELIS compared to paper registries, ease of implementation, and continuous technical support have facilitated the routine use of OpenELIS. In addition, an experienced software development team designed various functional and interface features of the software specifically intended to improve data quality, for example, drop-down menus for fast and accurate selections and automatic data completeness and validity checks. The decade-long development and implementation of OpenELIS in Côte d’Ivoire has been open-source, iterative, and collaborative so that feedback from users, laboratory managers, and governing bodies was directly incorporated into the software to match the needs of the users [15].

Our study provides evidence for improving laboratory data quality, which is important for the improvement of clinical care and treatment. A study from Kenya identified that reduced turnaround time and timely communication of laboratory test results are some of the key factors for improving HIV treatment retention [12].

In addition, in terms of patient safety, LISs reduced the probability of errors in patient identification and subsequent inappropriate treatment by reducing manual clerical work through automation and user-friendly widgets [10]. A survey of clinicians in Malawi revealed that laboratories’ poor documentation of test results was one of the reasons for having little trust in laboratory capability and not frequently using laboratory test results in patient management [13]. Last but not least, LISs can also improve disease surveillance by making it easier to record accurate demographic data that inform disease classification, assessment of population-specific rates, and contact tracing for infectious diseases [11], as well as to report laboratory-confirmed diagnoses to surveillance systems that allow more accurate estimation of disease burdens [14].

This evaluation has many strengths and some limitations. The use of multiple data points before and after OpenELIS helps capture the changes in time-varying variables through the data we used to generate the pre- and posttrends. However, since all sites that have been using OpenELIS, including the ones sampled in this evaluation, were purposefully selected due to their higher capacity, they might also have had a higher probability of adhering to the OpenELIS user instructions. The regression models did not specifically account for potential confounding effects from the COVID-19 pandemic and related control measures that were enforced from March to July 2020 mostly in the Greater Abidjan area in Côte d’Ivoire [40]. However, since the percentage of weekly data points occurring after March 2020 was low (3.3% for timeliness, 7.0% for completeness, and 7.7% for validity), and most of the data points came outside Abidjan, we thought that the influence of the pandemic on our outcomes was low. Furthermore, the 3 outcomes were proportions of the total testing data, so the estimates were less likely to be influenced by fluctuations in total testing volume due to the pandemic. Nonetheless, we still included the total number of tests performed by a laboratory as weight in the regression model. The results may be generalizable to remaining clinical laboratories in Côte d’Ivoire that are at the same tiers in the health system as the sites sampled in this study, but generalizability to laboratories in other LMICs needs further studies to demonstrate.

The implementation of an LIS can have a significant effect on improving data quality in LMICs. This study found that the implementation of OpenELIS led to improved timeliness and completeness of laboratory data. The system also facilitated better data availability compared to paper registries, enabling health care providers to make informed decisions based on accurate and up-to-date information.

The effectiveness of an LIS depends on many factors beyond the capabilities of the LIS itself, including systematic planning for adoption, supportive technical and implementation staff, attitudes of policy makers and users, presence of supportive policies, financial resources, infrastructure, and organizational and individual readiness to adopt innovations. Future research is needed to explore the facilitators and barriers to implementing health information systems such as an LIS in LMICs.