The Surveillance Outbreak Response Management and Analysis System (SORMAS): Digital Health Global Goods Maturity Assessment

Introduction

Overview

Digital health is a dynamic field with a rapidly growing number of initiatives and tools, many of which operate in certain geographic areas and for a limited period [1]. This lack of sustainability may be due to a variety of reasons, including the lack of integration in standard frameworks, lack of diverse and continuous donor support, lack of serving the objectives, and limited generalizability of its application among others [2-4]. In the field of public health, funding for such initiatives relies mostly on scarce public resources. Many such parallel initiatives do not seem to be mature enough to survive their pilot phase, which makes it particularly unfortunate. It is in this context that the concept of global goods maturity is gaining importance [5,6].

Global Goods

The concept of global goods stems from guidelines influencing health policies to support technologies that are meant to assist government agencies and policy makers in launching, scaling, and sustaining digital health innovations [7]. Different ministries of health convened advisory committees, including members of the digital health ecosystem and new multifaceted enterprises such as the Digital Impact Alliance and Digital Square [7]. These included global agencies, governments, philanthropies, funders, and academics to improve health data through shared investments in global goods and to fast-track the development and scale-up of successful digital health solutions to campaign for improvements in the facilitating environment for digital health [6,8]. To address the current lack of donor coordination, the committees created a framework to allow shared investments in specific countries and global goods. Health strengthening through enhancing country information systems would contribute to better decision making and, ultimately, better health [6]. The digital investment principles state that the funding donors within countries who are looking to prioritize their needs to improve the health of populations should align their resources around scalable, sustainable, accessible, interoperable, and evidence-based digital health global goods [6].

Surveillance Outbreak Response Management and Analysis System

The Surveillance Outbreak Response Management and Analysis System (SORMAS) is an open-source mobile and web application software that we developed to enable health workers to notify health departments about new cases of epidemic-prone diseases, detect outbreaks, and manage outbreak response at the same time. SORMAS is a management process system that supports supervisors to validate cases and control the spread of disease. As a multifunctional software, it can be used for case surveillance, laboratory data management, contact tracing, and disease detection to prevent and manage outbreaks that may occur. SORMAS also uses a bidirectional information exchange synchronizing user requests as well as sending feedback to the different users within the existing surveillance system [9-11].

Digital Square Global Goods Maturity Model

Digital Square, which is an initiative aimed at coordinating international efforts to develop and broadly share useful, free, and open-source digital tools, included 18 mobile health (mHealth) tools into the database of digital health software referred to as “global goods software” [12,13]. Digital Square developed a global good maturity model (GGMM) for digital health tools, which engages the digital health community to identify areas of investments for global goods [12,13]. We identified the GGMM to be a suitable concept in assessing the maturity of SORMAS for the following reasons: (1) GGMM has a particular focus on health software and on that being used in low-resource settings [12,13], (2) the objectives of GGMM matches well with the mission of SORMAS [12,13], and (3) the scope of most of the tools included in the GGMM guidebook fits that of SORMAS [12,13].

The following key concepts are prevalent throughout the study in determining the maturity of SORMAS according to the GGMM: global utility, community support, and software maturity. The objective of this study was to assess the level of global goods maturity that SORMAS has attained using the GGMM version 1.0.

Methods

We applied the GGMM version 1.0 on SORMAS from November 2017 until October 2019 to assess the level of global goods maturity that it has attained [13]. The GGMM contains 15 subindicators grouped into the following 3 core indicators: (1) global utility, (2) community support, and (3) software maturity. Each subindicator is divided into 3 possible values from −1 for “low,” 0 for “medium,” to 1 for “high.” Each value contains a definition that is listed in Table 1. Given that the GGMM scoring values range from negative to positive values, we applied the following transformation to allow percentage values for summary scores, as shown in equation (1), where, MS=maturity score for each core indicator, Si=vector containing scores of the subindicators, and i= 1,..., 5:

MS=5 * [MEAN (S_i)] + 5 (1)

We put together a GGMM assessment group consisting of a public health expert from the Nigerian Centre for Disease Control, a medical epidemiologist, an international health expert, an information technology specialist and a statistician from the Helmholtz Centre for Infections Research, and 2 software engineers from Symeda, a company for developing health software. This group periodically assessed the completion of SORMAS in the 15 subindicators. Subsequently, the software roadmap and the work plan of SORMAS was reprioritized to dedicate resources to and obtain progress in those subindicators that did not reach a full score. To reduce selection bias, as members of the assessment group were part of the development and deployment of SORMAS, and to reduce the potential partiality resulting from it, we also asked 2 external experts to review the findings made by the assessment group. These external experts had contributed to the development of the GGMM and were not involved in the development of SORMAS.

Results

In November 2017, SORMAS had migrated from a tabletop pilot version to a real-life deployed open-source software and was deployed in 8 federal states and 33 local government areas (LGAs) during the monkeypox outbreak. From September 2016 until November 2017, SORMAS was piloted in 1 state in 2 LGAs and 85 health facilities [14,15]. By February 2018, SORMAS was deployed in 3 additional federal states (71 LGAs) for meningitis outbreak. In March and April 2018, SORMAS was further deployed in 3 additional states (49 LGAs; 5 health facilities) for the Lassa fever outbreak. As of October 2019, SORMAS has been fully established for all epidemic-prone diseases in 15 federal states (including the federal capital), 287 LGAs, 37 health facilities, and approximately 700 users covering a population of 75 million. SORMAS managed multiple outbreaks simultaneously across the country. The assessment of SORMAS applying the GGMM showed that SORMAS had a 10-point score each in global utility, community support, and software maturity (see Table 2).

Discussion

Principal Findings

SORMAS has reached the full score of the GGMM. The process from the decision to migrate a prototype based on a Systems, Applications, and Products proprietary technology stack to open-source software in 2016 until the accomplishment of the full score lasted 3 years [10]. There is no registry or publicly available documentation of tools that have applied the GGMM assessment except the District Health Information System 2 [17]. Of all the mHealth tools selected as global goods software listed in the GGMM guidebook, none have accomplished over 90% of the full score nor are we aware of any other tool that has obtained it [13]. Thus, it appears very likely that SORMAS may be the first and, so far, only digital tool that has reached the full score of the GGMM. SORMAS has multiple revenue streams and funding mechanisms that have been enabling progress in its development since 2016. SORMAS requires funding for software adaptations to country-specific requests, personnel for training, and supervision and maintenance of the tool.

The outline and the scoring principle of the GGMM version 1.0 was easy to apply. However, there were some limitations concerning the clarity of the definition of each of the 3 possible criteria of the subindicators. Many of those definitions contained a combination of several contextually independent items, for which the wording did not clearly distinguish between “AND” or “OR” combinations. Some definitions left room for interpretation, which may be necessary for some but also be too ambiguous in other situations. The 30 subindicators grouped into 3 core indicators all have equal weights keeping the model and its handling simple.

On the other hand, it may also not represent the difference in importance that different indicators may have. For example, it may be considered essential for global goods maturity to have repetitive external security tests implemented than multilinguarity. Once the full score has been accomplished, these differences are no longer relevant. However, as long as the total score is only partially completed, the quantitative value may be very misleading in the attempt to compare tools. A way to improve this dilemma would be to add weights to each indicator. A more natural way would be to discourage the computation of proportional completion and to categorically apply a dichotomous all-or-nothing principle, by which all requirements have been either fully fulfilled or not fulfilled [18]. Some indicators address items that may develop into two directions. For example, the number of countries using the tool may not always diminish but also increase, or an external penetration test may not be repeated as required. It may increase scaling if those subindicators that do require a regular renewal or update be explicitly highlighted to facilitate monitoring of the status. Another difficulty appeared with the subindicator for global utility, “Source Code Accessibility,” as it was not entirely clear what “new modules” and “functionality” meant in the context of this model: In contrast to modular systems, some mHealth apps are built specifically to deliver a ready-to-go solution for a specific task, and they are defined by external standards that govern how the internal processes of the end users work. In such a situation, the requirement for facilitating options for developing new modules and functionalities without prematurely forking the code may not be applicable.

For the time being, the GGMM serves as a self-assessment system. In that, the authors can confirm that in the case of SORMAS, it has dramatically guided the prioritization and acceleration of its development. It appears to be the main objective of the model. On the other hand, self-assessment also allows for a certain degree of subjectivity or bias [19]. To reduce this risk, we contacted 2 external experts who were not involved in the development and deployment of SORMAS but were instead instrumental in the development of the GGMM. Through this we aimed to reduce conflicts of interest and maximize the expertise on SORMAS as well as on the interpretation of the GGMM. We believe that inviting independent external experts in reviewing this assessment may be a model for other projects and tools to apply the GGMM assessment. We do not entirely recommend an external assessment, as practiced in many accreditation procedures, because it carries the risk of creating a business that will only but draw resources needed for the actual development of the tools.

Conclusions

To our knowledge, SORMAS is the first electronic health tool for disease surveillance, and also the first outbreak response management tool, that has reached the full score (100%) of the GGMM. The GGMM is clear in most of its definitions and is easy to apply for self-assessment, although some indicators require more resources for completion than others. Some conceptual modifications would allow for further improvements to the system. Nevertheless, it already has a supportive effect on developing software toward global goods maturity.