Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 29.04.20 in Vol 6, No 2 (2020): Apr-Jun

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/15860, first published Sep 12, 2019.

This paper is in the following e-collection/theme issue:

    Original Paper

    The Surveillance Outbreak Response Management and Analysis System (SORMAS): Digital Health Global Goods Maturity Assessment

    1Helmholtz Centre for Infection Research, Department of Epidemiology, Braunschweig, Germany

    2Hannover Medical School (MHH), Hannover, Germany

    3Digital Square, PATH, Chapel Hill, North Carolina, North Carolina, NC, United States

    4Symeda GmbH, Braunschweig, Germany

    5Nigeria Center for Disease Control, Abuja, Nigeria

    Corresponding Author:

    Daniel Tom-Aba, BScCompSc, MPH

    Helmholtz Centre for Infection Research

    Department of Epidemiology

    Inhoffenstrasse 7

    Braunschweig, 38124

    Germany

    Phone: 49 01739136081

    Email: daniel.tom-aba@helmholtz-hzi.de


    ABSTRACT

    Background: Digital health is a dynamic field that has been generating a large number of tools; many of these tools do not have the level of maturity required to function in a sustainable model. It is in this context that the concept of global goods maturity is gaining importance. Digital Square developed a global good maturity model (GGMM) for digital health tools, which engages the digital health community to identify areas of investment for global goods. The Surveillance Outbreak Response Management and Analysis System (SORMAS) is an open-source mobile and web application software that we developed to enable health workers to notify health departments about new cases of epidemic-prone diseases, detect outbreaks, and simultaneously manage outbreak response.

    Objective: The objective of this study was to evaluate the maturity of SORMAS using Digital Square’s GGMM and to describe the applicability of the GGMM on the use case of SORMAS and identify opportunities for system improvements.

    Methods: We evaluated SORMAS using the GGMM version 1.0 indicators to measure its development. SORMAS was scored based on all the GGMM indicator scores. We described how we used the GGMM to guide the development of SORMAS during the study period. GGMM contains 15 subindicators grouped into the following core indicators: (1) global utility, (2) community support, and (3) software maturity.

    Results: The assessment of SORMAS through the GGMM from November 2017 to October 2019 resulted in full completion of all subscores (10/30, (33%) in 2017; 21/30, (70%) in 2018; and 30/30, (100%) in 2019). SORMAS reached the full score of the GGMM for digital health software tools by accomplishing all 10 points for each of the 3 indicators on global utility, community support, and software maturity.

    Conclusions: To our knowledge, SORMAS is the first electronic health tool for disease surveillance, and also the first outbreak response management tool, that has achieved a 100% score. Although some conceptual changes would allow for further improvements to the system, the GGMM already has a robust supportive effect on developing software toward global goods maturity.

    JMIR Public Health Surveill 2020;6(2):e15860

    doi:10.2196/15860

    KEYWORDS



    Introduction

    Overview

    Digital health is a dynamic field with a rapidly growing number of initiatives and tools, many of which operate in certain geographic areas and for a limited period [1]. This lack of sustainability may be due to a variety of reasons, including the lack of integration in standard frameworks, lack of diverse and continuous donor support, lack of serving the objectives, and limited generalizability of its application among others [2-4]. In the field of public health, funding for such initiatives relies mostly on scarce public resources. Many such parallel initiatives do not seem to be mature enough to survive their pilot phase, which makes it particularly unfortunate. It is in this context that the concept of global goods maturity is gaining importance [5,6].

    Global Goods

    The concept of global goods stems from guidelines influencing health policies to support technologies that are meant to assist government agencies and policy makers in launching, scaling, and sustaining digital health innovations [7]. Different ministries of health convened advisory committees, including members of the digital health ecosystem and new multifaceted enterprises such as the Digital Impact Alliance and Digital Square [7]. These included global agencies, governments, philanthropies, funders, and academics to improve health data through shared investments in global goods and to fast-track the development and scale-up of successful digital health solutions to campaign for improvements in the facilitating environment for digital health [6,8]. To address the current lack of donor coordination, the committees created a framework to allow shared investments in specific countries and global goods. Health strengthening through enhancing country information systems would contribute to better decision making and, ultimately, better health [6]. The digital investment principles state that the funding donors within countries who are looking to prioritize their needs to improve the health of populations should align their resources around scalable, sustainable, accessible, interoperable, and evidence-based digital health global goods [6].

    Surveillance Outbreak Response Management and Analysis System

    The Surveillance Outbreak Response Management and Analysis System (SORMAS) is an open-source mobile and web application software that we developed to enable health workers to notify health departments about new cases of epidemic-prone diseases, detect outbreaks, and manage outbreak response at the same time. SORMAS is a management process system that supports supervisors to validate cases and control the spread of disease. As a multifunctional software, it can be used for case surveillance, laboratory data management, contact tracing, and disease detection to prevent and manage outbreaks that may occur. SORMAS also uses a bidirectional information exchange synchronizing user requests as well as sending feedback to the different users within the existing surveillance system [9-11].

    Digital Square Global Goods Maturity Model

    Digital Square, which is an initiative aimed at coordinating international efforts to develop and broadly share useful, free, and open-source digital tools, included 18 mobile health (mHealth) tools into the database of digital health software referred to as “global goods software” [12,13]. Digital Square developed a global good maturity model (GGMM) for digital health tools, which engages the digital health community to identify areas of investments for global goods [12,13]. We identified the GGMM to be a suitable concept in assessing the maturity of SORMAS for the following reasons: (1) GGMM has a particular focus on health software and on that being used in low-resource settings [12,13], (2) the objectives of GGMM matches well with the mission of SORMAS [12,13], and (3) the scope of most of the tools included in the GGMM guidebook fits that of SORMAS [12,13].

    The following key concepts are prevalent throughout the study in determining the maturity of SORMAS according to the GGMM: global utility, community support, and software maturity. The objective of this study was to assess the level of global goods maturity that SORMAS has attained using the GGMM version 1.0.


    Methods

    We applied the GGMM version 1.0 on SORMAS from November 2017 until October 2019 to assess the level of global goods maturity that it has attained [13]. The GGMM contains 15 subindicators grouped into the following 3 core indicators: (1) global utility, (2) community support, and (3) software maturity. Each subindicator is divided into 3 possible values from −1 for “low,” 0 for “medium,” to 1 for “high.” Each value contains a definition that is listed in Table 1. Given that the GGMM scoring values range from negative to positive values, we applied the following transformation to allow percentage values for summary scores, as shown in equation (1), where, MS=maturity score for each core indicator, Si=vector containing scores of the subindicators, and i= 1,..., 5:

    MS=5 * [MEAN (Si)] + 5 (1)
    Table 1. Global good maturity model 1.0 guideline of digital intervention tools containing 15 subindicators from global utility, community support, and software maturity global indicators.
    View this table

    We put together a GGMM assessment group consisting of a public health expert from the Nigerian Centre for Disease Control, a medical epidemiologist, an international health expert, an information technology specialist and a statistician from the Helmholtz Centre for Infections Research, and 2 software engineers from Symeda, a company for developing health software. This group periodically assessed the completion of SORMAS in the 15 subindicators. Subsequently, the software roadmap and the work plan of SORMAS was reprioritized to dedicate resources to and obtain progress in those subindicators that did not reach a full score. To reduce selection bias, as members of the assessment group were part of the development and deployment of SORMAS, and to reduce the potential partiality resulting from it, we also asked 2 external experts to review the findings made by the assessment group. These external experts had contributed to the development of the GGMM and were not involved in the development of SORMAS.


    Results

    In November 2017, SORMAS had migrated from a tabletop pilot version to a real-life deployed open-source software and was deployed in 8 federal states and 33 local government areas (LGAs) during the monkeypox outbreak. From September 2016 until November 2017, SORMAS was piloted in 1 state in 2 LGAs and 85 health facilities [14,15]. By February 2018, SORMAS was deployed in 3 additional federal states (71 LGAs) for meningitis outbreak. In March and April 2018, SORMAS was further deployed in 3 additional states (49 LGAs; 5 health facilities) for the Lassa fever outbreak. As of October 2019, SORMAS has been fully established for all epidemic-prone diseases in 15 federal states (including the federal capital), 287 LGAs, 37 health facilities, and approximately 700 users covering a population of 75 million. SORMAS managed multiple outbreaks simultaneously across the country. The assessment of SORMAS applying the GGMM showed that SORMAS had a 10-point score each in global utility, community support, and software maturity (see Table 2).

    Table 2. The distribution of subindicator mean scores among the 3 core indicators (global utility, community support, and software maturity) for Surveillance Outbreak Response Management and Analysis System development from November 2017 to October 2019.
    View this table

    Discussion

    Principal Findings

    SORMAS has reached the full score of the GGMM. The process from the decision to migrate a prototype based on a Systems, Applications, and Products proprietary technology stack to open-source software in 2016 until the accomplishment of the full score lasted 3 years [10]. There is no registry or publicly available documentation of tools that have applied the GGMM assessment except the District Health Information System 2 [17]. Of all the mHealth tools selected as global goods software listed in the GGMM guidebook, none have accomplished over 90% of the full score nor are we aware of any other tool that has obtained it [13]. Thus, it appears very likely that SORMAS may be the first and, so far, only digital tool that has reached the full score of the GGMM. SORMAS has multiple revenue streams and funding mechanisms that have been enabling progress in its development since 2016. SORMAS requires funding for software adaptations to country-specific requests, personnel for training, and supervision and maintenance of the tool.

    The outline and the scoring principle of the GGMM version 1.0 was easy to apply. However, there were some limitations concerning the clarity of the definition of each of the 3 possible criteria of the subindicators. Many of those definitions contained a combination of several contextually independent items, for which the wording did not clearly distinguish between “AND” or “OR” combinations. Some definitions left room for interpretation, which may be necessary for some but also be too ambiguous in other situations. The 30 subindicators grouped into 3 core indicators all have equal weights keeping the model and its handling simple.

    On the other hand, it may also not represent the difference in importance that different indicators may have. For example, it may be considered essential for global goods maturity to have repetitive external security tests implemented than multilinguarity. Once the full score has been accomplished, these differences are no longer relevant. However, as long as the total score is only partially completed, the quantitative value may be very misleading in the attempt to compare tools. A way to improve this dilemma would be to add weights to each indicator. A more natural way would be to discourage the computation of proportional completion and to categorically apply a dichotomous all-or-nothing principle, by which all requirements have been either fully fulfilled or not fulfilled [18]. Some indicators address items that may develop into two directions. For example, the number of countries using the tool may not always diminish but also increase, or an external penetration test may not be repeated as required. It may increase scaling if those subindicators that do require a regular renewal or update be explicitly highlighted to facilitate monitoring of the status. Another difficulty appeared with the subindicator for global utility, “Source Code Accessibility,” as it was not entirely clear what “new modules” and “functionality” meant in the context of this model: In contrast to modular systems, some mHealth apps are built specifically to deliver a ready-to-go solution for a specific task, and they are defined by external standards that govern how the internal processes of the end users work. In such a situation, the requirement for facilitating options for developing new modules and functionalities without prematurely forking the code may not be applicable.

    For the time being, the GGMM serves as a self-assessment system. In that, the authors can confirm that in the case of SORMAS, it has dramatically guided the prioritization and acceleration of its development. It appears to be the main objective of the model. On the other hand, self-assessment also allows for a certain degree of subjectivity or bias [19]. To reduce this risk, we contacted 2 external experts who were not involved in the development and deployment of SORMAS but were instead instrumental in the development of the GGMM. Through this we aimed to reduce conflicts of interest and maximize the expertise on SORMAS as well as on the interpretation of the GGMM. We believe that inviting independent external experts in reviewing this assessment may be a model for other projects and tools to apply the GGMM assessment. We do not entirely recommend an external assessment, as practiced in many accreditation procedures, because it carries the risk of creating a business that will only but draw resources needed for the actual development of the tools.

    Conclusions

    To our knowledge, SORMAS is the first electronic health tool for disease surveillance, and also the first outbreak response management tool, that has reached the full score (100%) of the GGMM. The GGMM is clear in most of its definitions and is easy to apply for self-assessment, although some indicators require more resources for completion than others. Some conceptual modifications would allow for further improvements to the system. Nevertheless, it already has a supportive effect on developing software toward global goods maturity.

    Acknowledgments

    The authors would like to thank BMZ, Federal Ministry of Economic Cooperation and Development; European Union, via Deutsche Gesellschaft für Internationale Zusammenarbeit, Deutschen Zentrum für Infektionsforschung for funding the development of SORMAS since 2015.

    Authors' Contributions

    DT, BS, and GK searched, selected, and extracted data based on the criteria. DT conceptualized the first draft and GK analyzed and interpreted the results of the data. DT, GK, BS, CA, and JD reviewed the GGMM criteria for SORMAS development. MW and MS contributed to the development of SORMAS software. CL and CF represented Digital Square and the initiative that convened the initial creation of the GGMM version 1.0 and assisted in the SORMAS GGMM assessment. GK initiated the study approach and supervised all steps of the study. All authors revised and contributed to the final manuscript.

    Conflicts of Interest

    None declared.

    References

    1. Tom-Aba D, Nguku PM, Arinze CC, Krause G. Assessing the concepts and designs of 58 mobile apps for the management of the 2014-2015 West Africa Ebola outbreak: systematic review. JMIR Public Health Surveill 2018 Oct 29;4(4):e68 [FREE Full text] [CrossRef] [Medline]
    2. AbouZahr C, Boerma T. Health information systems: the foundations of public health. Bull World Health Organ 2005 Aug;83(8):578-583 [FREE Full text] [Medline]
    3. Free C, Phillips G, Watson L, Galli L, Felix L, Edwards P, et al. The effectiveness of mobile-health technologies to improve health care service delivery processes: a systematic review and meta-analysis. PLoS Med 2013;10(1):e1001363 [FREE Full text] [CrossRef] [Medline]
    4. Qiang CZ, Yamamichi M, Hausman V, Altman D. World Bank. Washington, DC: World Bank; 2011 Dec. Mobile Applications for the Health Sector   URL: http:/​/siteresources.​worldbank.org/​INFORMATIONANDCOMMUNICATIONANDTECHNOLOGIES/​Resources/​mHealth_report.​pdf [accessed 2020-03-03]
    5. Labrique AB, Wadhwani C, Williams KA, Lamptey P, Hesp C, Luk R, et al. Best practices in scaling digital health in low and middle income countries. Global Health 2018 Nov 3;14(1):103 [FREE Full text] [CrossRef] [Medline]
    6. Digital Impact Alliance (DIAL). 2017.   URL: https://digitalimpactalliance.org/ [accessed 2019-10-31]
    7. Health Data Collaborative. What We Do: Better Data. Better Health   URL: https://www.healthdatacollaborative.org/what-we-do/ [accessed 2017-10-20]
    8. PATH. 2017 Sep 28. Digital Health: New Global Initiative Will Connect the World for Better Health Through Digital Technology   URL: http://www.path.org/news/press-room/846/ [accessed 2019-10-31]
    9. Fähnrich C, Denecke K, Adeoye OO, Benzler J, Claus H, Kirchner G, et al. Surveillance and Outbreak Response Management System (SORMAS) to support the control of the Ebola virus disease outbreak in West Africa. Euro Surveill 2015 Mar 26;20(12):pii: 21071 [FREE Full text] [CrossRef] [Medline]
    10. Perscheid C, Benzler J, Hermann C, Janke M, Moyer D, Laedtke T, et al. Ebola outbreak containment: real-time task and resource coordination with SORMAS. Front ICT 2018;5:7. [CrossRef]
    11. Adeoye O, Tom-Aba D, Ameh C, Ojo O, Ilori E, Gidado S, et al. Implementing Surveillance and Outbreak Response Management and Analysis System (SORMAS) for public health in west Africa- lessons learnt and future direction. Int J Trop Dis Health 2017;22(2):1-17. [CrossRef]
    12. Digital Square. 2019 May. Global Goods Guidebook   URL: https://digitalsquare.org/global-goods-guidebook [accessed 1999-11-30]
    13. Digital Square Wiki. What are Global Goods   URL: https://wiki.digitalsquare.io/index.php/What_are_Global_Goods [accessed 1999-11-30]
    14. Yinka-Ogunleye A, Aruna O, Dalhat M, Ogoina D, McCollum A, Disu Y, CDC Monkeypox Outbreak Team. Outbreak of human monkeypox in Nigeria in 2017-18: a clinical and epidemiological report. Lancet Infect Dis 2019 Aug;19(8):872-879. [CrossRef] [Medline]
    15. Silenou BC, Tom-Aba D, Adeoye O, Arinze CC, Oyiri F, Suleman AK, et al. Use of surveillance outbreak response management and analysis system for human monkeypox outbreak, Nigeria, 2017-2019. Emerg Infect Dis 2020 Feb;26(2):345-349 [FREE Full text] [CrossRef] [Medline]
    16. Tom-Aba D, Toikkanen SE, Glöckner S, Adeoye O, Mall S, Fähnrich C, et al. User Evaluation Indicates High Quality of the Surveillance Outbreak Response Management and Analysis System (SORMAS) After Field Deployment in Nigeria in 2015 and 2018. Stud Health Technol Inform 2018;253:233-237. [Medline]
    17. District Health Information System. Digital Square. 2019. Global Goods Maturity Assessment   URL: https://applications.digitalsquare.io/notice-c/concept-note/76 [accessed 2020-02-20]
    18. Wing O, Huang JW. A computation model of parallel solution of linear equations. IEEE Trans Comput 1980;C-29(7):632-638. [CrossRef]
    19. Bernheim JL, Theuns P, Mazaheri M, Hofmans J, Fliege H, Rose M. The Potential of Anamnestic Comparative Self-Assessment (ACSA) to Reduce Bias in the Measurement of Subjective Well-Being. J Happiness Stud 2006;7(2):227-250. [CrossRef]


    Abbreviations

    GGMM: global good maturity model
    LGA: local government area
    mHealth: mobile health
    SORMAS: Surveillance and Outbreak Response Management and Analysis System


    Edited by G Eysenbach; submitted 12.09.19; peer-reviewed by A BenDor, G Akpan, S Maleghemi; comments to author 28.10.19; revised version received 06.12.19; accepted 27.01.20; published 29.04.20

    ©Daniel Tom-Aba, Bernard Chawo Silenou, Juliane Doerrbecker, Carl Fourie, Carl Leitner, Martin Wahnschaffe, Maté Strysewske, Chinedu Chukwujekwu Arinze, Gerard Krause. Originally published in JMIR Public Health and Surveillance (http://publichealth.jmir.org), 29.04.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on http://publichealth.jmir.org, as well as this copyright and license information must be included.