Published on 07.04.20 in Vol 6, No 2 (2020): Apr-Jun
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/16119, first published Sep 03, 2019.
Participatory Surveillance Based on Crowdsourcing During the Rio 2016 Olympic Games Using the Guardians of Health Platform: Descriptive Study
Background: With the evolution of digital media, areas such as public health are adding new platforms to complement traditional systems of epidemiological surveillance. Participatory surveillance and digital epidemiology have become innovative tools for the construction of epidemiological landscapes with citizens’ participation, improving traditional sources of information. Strategies such as these promote the timely detection of warning signs for outbreaks and epidemics in the region.
Objective: This study aims to describe the participatory surveillance platform Guardians of Health, which was used in a project conducted during the 2016 Olympic and Paralympic Games in Rio de Janeiro, Brazil, and officially used by the Brazilian Ministry of Health for the monitoring of outbreaks and epidemics.
Methods: This is a descriptive study carried out using secondary data from Guardians of Health available in a public digital repository. Based on syndromic signals, the information subsidy for decision making by policy makers and health managers becomes more dynamic and assertive. This type of information source can be used as an early route to understand the epidemiological scenario.
Results: The main result of this research was demonstrating the use of the participatory surveillance platform as an additional source of information for the epidemiological surveillance performed in Brazil during a mass gathering. The platform Guardians of Health had 7848 users who generated 12,746 reports about their health status. Among these reports, the following were identified: 161 users with diarrheal syndrome, 68 users with respiratory syndrome, and 145 users with rash syndrome.
Conclusions: It is hoped that epidemiological surveillance professionals, researchers, managers, and workers become aware of, and allow themselves to use, new tools that improve information management for decision making and knowledge production. This way, we may follow the path for a more intelligent, efficient, and pragmatic disease control system.
JMIR Public Health Surveill 2020;6(2):e16119
Participatory surveillance has been a reality in many parts of the world, improving traditional health surveillance systems and engaging the population to build epidemiological scenarios [- ]. The use of mobile devices to improve the data collection, processing, and analysis processes for epidemiology and surveillance has contributed to great advances in public health in the aspects of innovation and digital transformation in this area [ - ]. With the evolution and ubiquity of mobile devices and their operating systems, in addition to increasing digital inclusion and internet connectivity, research and collaborative strategies have been adopted to improve the quality of information generated in health, especially in the understanding of epidemiological patterns [ - ]. Strategies for monitoring respiratory, diarrheal, and rash syndromes due to arboviruses are examples of how to drive digital disease detection platforms to address the production of strategic information for health surveillance based on crowdsourcing in the American, European, African, and Asian continents; some platforms include Flu Near You, Influenza.Net, AfyaData, Vigilant-e, Saúde na Copa, and Guardians of Health [ , - ]. The use of crowdsource-based platforms has also been observed in foodborne disease surveillance, which has enabled the anticipation of disease outbreak detection and evaluation of policies for food safety, as is the case for the website iwaspoisoned.com [ ].
Participatory surveillance systems usually work in similar ways, where the user is able to make a self-report of symptoms periodically. The periodicity varies from daily to weekly frequency. The collected data—symptoms and geolocation—are sent to cloud-computing servers, where data points are processed and analyzed. From the extracted data, epidemiologists, researchers, data scientists, and government agents analyze the information identifying the distribution of people with symptoms during a certain time frame . Usually the approach adopted is syndromic (ie, the data collection is specific for symptoms that make up groups of diseases), thus calibrating the sensitivity and specificity of the systems. Some platforms, such as Saúde na Copa [ ], Flu Near You [ ], or FluTracking [ ], use engagement strategies to ensure the user's adherence for regular participation within the system. In the first example, gamification was used; during the 2014 World Cup, users could play a soccer-themed game, where evolution within the game was conditional on users’ health reports. In the second example, nudges, such as referring a friend, showing the number of active users by region, awarding heavy users with badges, push notifications, and email reminders, were used to encourage participation.
Parallel to the advances of participatory surveillance, some health outcomes still need more intelligent and agile monitoring, such as the case of mass gatherings. Mass gatherings are situations involving large numbers of people participating in a specific cause, planned or not, related to leisure, religion, politics, sports, and other reasons .
Leal Neto et al  pointed out the use of participatory surveillance in mass gatherings for the first time during the 2014 FIFA World Cup, where a mobile app based on crowdsourcing was developed to identify health threats and was officially used by the Brazilian Ministry of Health. With this experience, and with the aim of improvement, a new platform for participatory surveillance in mass gatherings was developed and adopted, this time focusing on the 2016 Olympic Games [ ]. Initiatives conducted by other countries during the 2016 Olympic Games were carried out, demonstrating the understanding of the importance of new participatory surveillance approaches using mobile devices to serve as an additional support for traditional health systems [ ]. In addition, this work pointed out the relevance of this approach in finding outbreaks in a faster way [ ].
This work aims to describe the participatory surveillance platform Guardians of Health, a project conducted during the 2016 Olympic and Paralympic Games in Rio de Janeiro, which is officially used by the Brazilian Ministry of Health for the monitoring of outbreaks and epidemics.
Skoll Global Threats Fund and Epitrack, with the support of the Brazilian Ministry of Health and the Pernambuco Research Support Foundation, developed a mobile app, a web app, and a dashboard platform to implement participatory surveillance in Brazil during the 2016 Olympic Games. The Rio de Janeiro-based project also included five other Brazilian cities that hosted events and games related to the 2016 Olympic Games: Manaus, Salvador, São Paulo, Brasilia, and Belo Horizonte. The platform was called Guardiões da Saúde (Guardians of Health in Portuguese). The study period was divided into a pre-event period (March 28-August 4, 2016) and an event and postevent period (August 5-October 26, 2016). The first period aligned with the time when the platform Guardians of Health was officially launched by the Brazilian Ministry of Health. The second period aligned with the occurrence of the 2016 Olympic and Paralympic Games and the 45 days following the Games.
The management structure of the platform worked across multiple centers: a base at the General Coordination of Surveillance and Response to Events of Public Health, Secretariat of Health Surveillance in the Ministry of Health in Brasilia; a base in Washington, DC, USA; and a base for development and support in Recife, Brazil.
The Guardians of Health platform was composed of three segments: (1) iOS and Android apps, (2) a web app, and (3) a dashboard. The mobile app was developed using the native technology of the respective operating systems: for Android, Java was used; for iOS, Objective-C and Swift were adopted. Users who registered with information about sex, age, and city were motivated daily—by push notification, gamification, and marketing—to report their health condition. Within the options of the report, the user was able to state whether he or she was well (ie, without symptoms) or ill, where the following list of symptoms was shown, asking the user to pick one or more: body pain, headache, joint pain, cough, sore throat, fever, shortness of breath, nausea and/or vomiting, diarrhea, itching, rash, red eyes, and bleeding. The symptoms were strategically defined from a syndromic approach with the purpose of the identification of diarrheal, respiratory, and rash syndromes (see). For this project, rash syndrome was defined as symptoms related to arboviruses—dengue, Zika, and chikungunya—and the rash (ie, exanthema) symptom was necessary. For diarrheal syndrome, the diarrhea symptom was mandatory. For respiratory syndrome, cough and fever were mandatory symptoms.
Registrations and self-reports were completed and sent to the database. Their geographic coordinates—latitude and longitude per Universal Transverse Mercator—were then collected, favoring the geolocation of the records within the app. For devices that did not allow location tracking, we used the proxy location of the users’ Internet Protocols (IPs). The list of symptoms was accompanied by three more questions: (1) Did the user have contact with anyone with these symptoms? This served to establish possible links for eventual chain of transmission, (2) Did the user seek a health service to understand the severity of reported symptoms? and (3) Did the user travel abroad? This served to determine the possible introduction of acquired disease outside Brazil.
The purpose of the dashboard was to serve as an early-warning alert system for possible syndromic clusters; epidemiological surveillance technicians used the dashboard to monitor epidemiological patterns and possible disease outbreaks. To this end, component scripts and algorithms were developed in R, version 3.2.4 (The R Foundation), which worked on the cloud using cronjobs—a time-based job scheduler—and triggered changes in epidemiological patterns with color codes and alert signals.
From data reported by users, a time series was created by counting the number of events per day in each location for each of the signs and/or symptoms. One of the main challenges was to choose a technique that fit count data, possibly inflated by zeros, that presented small mean events per day with great variability, as observed in Saúde na Copa , which served as a test set for the methodologies to be applied. Given the characteristics of the data described above—count data inflated by zeros with overdispersion (ie, variance greater than the mean)—traditional methods were not applied. For instance, the autoregressive integrated moving average model requires data with normal average distribution; it does not reach convergence when dealing with quantities with high exponential coefficients, with many zeros, or with very small quantities, in addition to a larger set of observations. Model-based methods also exhibit the same nonconvergence constraints, even if using negative Poisson or binomial regression. Some methods, such as cumulative sum and exponentially weighted moving average, are more complex and the detection techniques for average changes can be challenging for the type of system to be implemented.
For these reasons, we chose an ad hoc approach by constructing a strategy to monitor signals from the Guardians of Health using the locally estimated scatterplot smoothing (loess) function in R. This technique uses a semiparametric local regression method, which has a smoothing parameter that defines how the function will be adjusted to the observed data. In addition to the smoothing parameter, we can also manipulate the confidence interval of the smoothed function; both parameters are easy to understand and, together with a simple rule, we have created the algorithm described below. From the series of signals, a loess is defined at the points passed and an associated confidence interval is established. Thus, the data arriving at the processing server can be checked against the series, and when a value passes the upper interval as defined by the parameters, a yellow alert is triggered. If the next point continues to exceed the upper limit, an orange alert is triggered. If the limit is exceeded three or more times, a red alert is triggered.shows the prototype output and simulation from the algorithm that we used.
It is important to note that because loess is a local regression method, a slow increase in cases also leads to an increase, albeit with a delay, in the upper range. This allows a very gradual increase in reported cases to be viewed as an average process increase, such as more people entering the system, so we have an increase in reported cases. Another important point is that as we adjust the parameters, possibly having different values for each signal or for each syndrome, we can calibrate the alert. If it is very sensitive we can reduce the number of alerts, and if the trend is rising slowly we can choose to have an alarm with a shorter latent memory.
The whole system used an API developed using the Sails framework that communicates with representational state transfer technology API (REST API). The database model used was MongoDB and the system was hosted by Amazon Web Services.
|Symptom||Diarrheal syndrome||Respiratory syndrome||Arbovirus (ie, rash) syndrome|
|Shortness of breath||—||Optional||—|
|Nausea and/or vomiting||Optional||—||—|
Engagement: Acquisition, Adherence, and Permanence of Users in the Platform
One of the biggest challenges for digital platforms that require users to participate via crowdsourcing in information building is recruiting and engaging them to use the apps. There are a few examples of this recruitment and engagement, such as the Waze traffic app that has 65 million active users in more than 185 countries . In the case of health, numerous strategies described by Smolinski at al [ ] describe how participatory surveillance platforms seek to motivate and connect more and more with users. In the case of Guardians of Health, some strategies were developed to achieve good levels of acquisition, adherence, and engagement among users.
The distribution of the app was carried out by the Apple App Store and the Android Google Play Store. Within app store profiles, criteria that favored app search optimization were met, including using keywords and strategic terms that would improve the positioning of apps when users were looking for health-related terms. As users could also register on the web, elements of search engine optimization were implemented, with the aim of improving Google's search positioning with terms related to the health scenario. In-store ratings and comments were also monitored, where a team was responsible for responding to questions, comments, and criticisms in a timely manner, thereby by improving customer relationship management.
From the perspective of marketing as a way of acquiring users, vertical tactics of launch and media buzz were developed to target the knowledge and dissemination of the app to the regions that were associated with the 2016 Olympic Games. Press conferences, for spontaneous media generation, were given and app information was placed in Brazil's main offline and online media channels, press releases, and press kits, among others. Inbound and outbound marketing approaches were developed to target the most reach, impressions, and conversions of leads as well as users. For a marketing outbound approach, Facebook Ads, Google AdWords, YouTube ads, mobile ad networks, Google paid search, and Twitter campaigns were circulated, focusing mainly on the region of Rio de Janeiro where users were impacted by the media. For inbound marketing, blog posts were used for health issues with a high number of visits, directing readers to the distribution channels of the apps.
A gamification piece within the mobile app and the web app was developed, reinforcing the goal of user engagement in a systematic and recurring way. A game was developed into the app that consisted of a quiz with more than 300 questions about health promotion issues, disease prevention, and vector-borne diseases; this brought a health education component to the active users. The questions in the quiz were prepared by the Brazilian Ministry of Health. When answering the questions, the users were presented with information and curiosities about Olympic sports and led users through an Olympic journey, bringing the theme to the digital environment and building an imaginary mindset of the Olympic Games.
The apps also featured a health guide with information on arboviruses, travelers’ health, location of emergency room units, vaccines, useful telephone numbers, drugstore locations, basic health care recommendations, and prevention of sexually transmitted infections.
One of the limitations of crowdsourcing is the reliability of information coming from users. The validation of a possible health threat, such as the beginning of an outbreak or epidemic, is made based on groups of people reporting similar symptoms in near time and space. In the analyses performed, a spam classification was created, which included records with eight or more symptoms reported. It was agreed that reports with these characteristics would be removed from the analyses, as they were characterized as spam or noise within the registration database.
Another element that was considered in constructing the platform was the possibility of including secondary users nested within a primary user in Guardians of Health. In this way, a family could have one primary user and other family members added as secondary users to the account.
Data from the results of downloads, registrations, and user reports within the Guardians of Health platform were analyzed using RStudio and the Exploratory.io framework. Guardians of Health is an open source and open data project available at the platform’s project page on GitHub via Epitrack .
During the study period (see), the app was downloaded 59,312 times: 95.47% (56,628/59,312) on Android devices and 4.53% (2684/59,312) on iOS devices. These downloads generated 7848 users (13.23%), where 5987 users (10.09%) sent at least one report about their health status. Of this total, 76.37% (4572/5987) were users with Android devices and 19.21% (1150/5987) were users with iOS devices. Only 265 users of the platform out of 5987 (4.43%) came via the web app. At the end of the 2016 Olympic Games period, we saw that Android device users’ churn (ie, loss of users) was 68.66% (38,878/56,628) and iOS device users’ churn was 60.39% (1621/2684).
This universe of users generated a total of 12,743 reports; after the classification and filtering of spam, this resulted in 71.79% (9148/12,743) of valid reports. A total of 80.92% (7403/9148) of reports had no symptom status and 19.08% (1745/9148) of reports presented at least one symptom in the period studied.
Regarding the users’ demographic profiles, 60.13% (5501/9148) were male and 39.89% (3649/9148) were female. The users’ ages ranged from 8 to 97 years, with a median of 39 years and a mean of 40.39 years (SD 14.06). Based on gender and ethnic policies of the Brazilian Ministry of Health, the app included a question about the race or color of the user: 30.31% (2773/9148) declared themselves as white, 28.54% (2611/9148) selected black, 21.28% (1947/9148) selected yellow, 18.10% (1656/9148) selected brown, and 1.78% (163/9148) declared themselves as Indigenous. Most of the reports (4601/9148, 50.30%) came from the city of Rio de Janeiro. The city of São Paulo had the second-highest number of reports (937/9148, 10.24%). Less than 5% of the reports came from other cities that had Olympic Games events, for all cities combined. Even with the platform’s promotion focused on Rio de Janeiro and other host cities, 18.89% (1728/9148) of the reports came from other cities in Brazil and, in some cases, from other countries. The average number of reports per user was 1.54 (SD 3.18, IQR 0.0) and 97.72% (8939/9148) of reports were made by the main user of the account. The average number of reports per day was 45.06 (SD 54.52).
Regarding the Guardians of Health syndromic profile, 1.76% (161/9148) of the reports were classified as diarrheal syndrome, 1.59% (145/9148) were classified as rash syndrome by arboviruses, and 0.74% (68/9148) were classified as respiratory syndrome. The frequencies of the most-reported symptoms are reported inand the syndrome distribution is demonstrated in .
Regarding the auxiliary questions, 1.96% (179/9148) of the reports were from users who had contact with someone with any of the symptoms described in the list. Regarding those who reported having sought health services, 2.87% (263/9148) were observed by a heath care professional. In addition, 3.05% (279/9148) of the reports were made by users who had been out of the country during the previous 2 weeks at the time of the competition.shows the spatial distribution of users who were feeling ill, with some symptoms. - show the spatial distribution of reports that were compatible with diarrheal, respiratory, and rash syndromes, respectively.
Throughout the study period, whether during the pre-event period or during the 2016 Olympic Games, despite the evidence of reports compatible with the syndromes described above, there was no concentration of these reports in the same space and time; this excluded the possibility of the beginning of outbreaks, according to the information collected by the Guardians of Health. We have tested this using k-means and the Hartigan-Wong algorithm.
Regarding the results obtained via the engagement strategies, marketing campaigns were made through the study’s YouTube channel to deliver video campaigns about the app, which were viewed 253,061 times. Regarding Facebook, content-placement strategies on the study’s own fan page were adopted, leveraging 439 followers. However, Facebook also posted on partner pages, such Razões para Acreditar, which has 692,297 followers, with Guardians of Health content being disseminated through these vehicles. Regarding scores on app stores, the Guardians of Health app averaged 4.1 out of 5.0 in the Google Play Store and 3.0 out of 5.0 in the Apple App Store.
|Symptom||Number of reports (N=1746 users with at least one symptom), n (%)|
|Body pain||607 (34.77)|
|Joint pain||487 (27.89)|
|Sore throat||277 (15.86)|
|Shortness of breath||218 (12.76)|
|Red eyes||132 (7.56)|
Participatory surveillance continues to be an alternative for health services innovation in the digital age, spreading to various parts of the world and increasingly gaining strength as an additional platform to traditional epidemiological surveillance systems [, , ]. The use of this platform on a national scale in Brazil during the 2014 FIFA World Cup and the 2016 Olympic Games shows that the government is willing to take a new position regarding disease surveillance; combining tools can bring modernization and substantial improvements to the sensitivity and specificity of the production and consumer structures of strategic information in health surveillance.
Experiences using mobile devices take advantage of the population's access to smartphones; this facilitates the spreading of projects that are based on real-time data collection with remote transmission of information and dashboards to support decision making [, - ]. In the case of Guardians of Health, for the period of operation, the high number of downloads was not effectively converted into the creation of users, where 13.23% of all who downloaded the app became registered users. This behavior can be explained with some hypotheses, such as the innovative potential of the tool, invoking the criterion of deceptive growth [ ], that is, the incredulity of the population in something more modern than the traditional systems. The tone and sentiment observed in the comments at the virtual stores and on Facebook point to another hypothesis linked to political instability at that time in Brazil.
There was a majority of users who used the Android operating system compared to the iOS operating system. The direct explanation is the lower cost of smartphones with the Android operating system in Brazil, in keeping with the financial reality of the population. Gotz et al  questioned whether there was a difference in personality between the users of both systems, but no relevant results were found to support this. Therefore, it can be suggested that the economic factor involved in the acquisition cost of these devices is what generated the greater preference for Android, at least in Brazil. Data show that Brazil currently has 77.35 million smartphone users, where 92.6% are Android users [ ]. The low number of registrations and use via the web app implies that internet apps are more restricted to mobile devices, as compared to desktops or access by browsers. The churn achieved by the apps was within an expected value for apps in the health field, which is 75% [ ].
Using a rule to identify spam posts , 28.21% of reports that were not true or had a high chance of being fallacious were eliminated. Even with this withdrawal, obtaining 9148 reports generated a reasonable amount of data for analysis. There were differences between the demographic profiles identified in the Guardians of Health platform and the Saúde na Copa app, which is a participatory surveillance app used in Brazil during the 2014 World Cup. Guardians of Health showed an overlap of approximately 20 percentage points in relation to male users. There was a difference in the age of users as well, where Saúde na Copa had a smaller age range (12-77 years) than did Guardians of Health (8-97 years). One of the elements that can explain this difference is that Guardians of Health had the functionality of adding secondary users to a primary account. That is, an adult at the median age observed (39 years) could have children or the elderly as secondary users within their account. This may also explain the difference in means achieved between the World Cup experience and the Olympic Games [ ].
The population profile according to race or color was a variable used for the first time in any participatory surveillance platforms, worldwide; this prevented a comparison of this profile with other regions, even aiming to raise discussions about access to technology according to race or color. However, due to the fact that Brazil has a well-known population with a majority of Europeans and Africans, while preserving Indigenous characteristics in several regions of the country, the figures presented reflect a reality of race diversity at the national level, even those who declare themselves to have represented only 1.79% of the reports.
Most of the Olympic and Paralympic competitions were held in Rio de Janeiro, explaining that most of the reports came from this locality. One situation that has been repeated in relation to the Saúde na Copa app  is the demonstration of the potential for scale and dissemination of a participatory surveillance platform for mobile devices. In the Guardians of Health platform, 30.83% of the reports came from Brazilian cities or from foreign territories—the minority—that were not the headquarters of any Olympic event.
The results on the captured syndromic profiles (ie, reports that were compatible with the rules of a priori defined syndromes) were below those observed in previous experiences during mass events using participatory surveillance. However, they still show great potential for use because of their sensitivity in locally identifying concentrations of reports with similar characteristics of symptoms. Fortunately, there were no outbreaks detected by the app and this was corroborated with official information from the Brazilian Ministry of Health, assuming validation of the tool's potential in the timely detection of health threats.
Unlike the Saúde na Copa app, which had peaks of participation during matches of the Brazilian soccer team, the Olympics had a diversity of sports activities; it was possible that the peaks in the number of participating users and completion of reports were related to the days when the marketing campaigns for acquisition and adherence of users took place.
Mass gatherings continue to be sensitive situations in health management, due to the pressure caused within local systems; a sudden increase in demand of resources with a structure unable to keep up with the scale of supply; as well as the epidemiological risks of introducing eradicated, new, or nonendemic diseases as well as controlled diseases into the national context. Initiatives like Guardians of Health help to track risk factors for epidemics and diseases outbreaks. Even when these risks occur during mass gatherings, such platforms are able to minimize these health threats.
Engaging users on platforms such as this remains one of the puzzling and challenging issues in terms of participatory surveillance. Building on the importance of understanding health information around them has been a quest for various groups working with participatory surveillance around the world; despite experiences with interesting success (ie, FluTracking ), a replicable path has not been found, making this a negative aspect in applying a participatory approach [ - ].
Another sensitive point that comes up on the list of challenges is the role of government and its agents as users and system managers. The importance of a prospective mindset is vital to foster abundant, innovation-oriented thinking, in order to improve and sustain these initiatives. It would be ideal for each experience, whether during the World Cup or the Olympic Games, to have, at minimum, a permanent structure of management, development, support, and dissemination that would favor growth more and more. However, these skewed interests and lack of agile capacity, which are needed to structure the sectors that managed to maintain projects like this, are also characterized as enigmatic challenges; a solution needs to be found to articulate and implement these experiences as a permanent part of innovative health policy.
In this scenario of the information capture flow for epidemiological surveillance, sick individuals are only counted by the health system upon entry to the system, which is then notified when necessary. However, the time between illness and demand for health care, assuming the health facility would report all relevant cases, demonstrates the fragility of traditional systems regarding the timely identification of diseases that can impact public health in the form of outbreaks and epidemics. One way to fill this gap between illness from a disease with outbreak potential and the record of it is the use of technologies and strategies, such as participatory surveillance; these empower citizens by making them an active part of joint information building, which contributes to the epidemiological setting of their community or region and rescues the precepts of social control widely debated in the Sistema Único de Saúde, the Brazilian National Health System.
Reaffirming the debate [, ], the explicit evidence that disruptive innovations in public health are far more present in our national context than we imagine, because of their potential exponential growth, urges research and services to consider this new movement in their collective health practices. In this way, Sistema Único de Saúde can not only follow but can become an effective actor in this rapid transformation in the use of information to improve the quality of life for all citizens.
Many other participatory surveillance strategies keep appearing across the globe, using the same mindset where people are the primary data source, contributing to build epidemiological settings with crowdsourcing [- ]. On the other hand, nontraditional approaches for health communication should be considered to work alongside traditional methods in order to increase the range of digital health, for instance, the use of YouTube to spread health education content [ , ].
Digital transformation is a fact; it is no longer a futuristic element, but has become today's reality. The world is changing fast and recognizing this transformation is essential to face today’s challenges. The struggle against those who continue to ignore this change encumbers the process of transformation, holding all those working in the context of public health hostage to obsolescence. It is hoped that professionals, new and old, as well as researchers, managers, and workers involved in epidemiological surveillance become aware and allow themselves to implement new tools that improve information management for decision making and knowledge production.
ML and MS were funded by Ending Pandemics and participated in the study when it was rolled out during the Olympic Games. Beyond that, the funder provided support in the form of salaries for ML and MS.
OLN, JA, and MNS were responsible for study conceptualization, data analysis, and writing of the paper. ML and MS participated in the study when it was rolled out during the Olympic Games.
Conflicts of Interest
In order to make clear the commercial affiliations of some of the authors, we would like to declare that OLN, JA, and MNS worked on the project as key professionals in the development and implementation of the Guardians of Health platform.
- Ginsberg J, Mohebbi MH, Patel RS, Brammer L, Smolinski MS, Brilliant L. Detecting influenza epidemics using search engine query data. Nature 2009 Feb 19;457(7232):1012-1014. [CrossRef] [Medline]
- Yuan Q, Nsoesie EO, Lv B, Peng G, Chunara R, Brownstein JS. Monitoring influenza epidemics in China with search query from Baidu. PLoS One 2013;8(5):e64323 [FREE Full text] [CrossRef] [Medline]
- Wójcik OP, Brownstein JS, Chunara R, Johansson MA. Public health for the people: Participatory infectious disease surveillance in the digital age. Emerg Themes Epidemiol 2014;11:7 [FREE Full text] [CrossRef] [Medline]
- Leal-Neto OB, Dimech GS, Libel M, Oliveira W, Ferreira JP. Digital disease detection and participatory surveillance: Overview and perspectives for Brazil. Rev Saude Publica 2016;50:17. [CrossRef] [Medline]
- Meyers DJ, Ozonoff A, Baruwal A, Pande S, Harsha A, Sharma R, et al. Combining healthcare-based and participatory approaches to surveillance: Trends in diarrheal and respiratory conditions collected by a mobile phone system by community health workers in rural Nepal. PLoS One 2016;11(4):e0152738 [FREE Full text] [CrossRef] [Medline]
- Bach M, Jordan S, Hartung S, Santos-Hövener C, Wright MT. Participatory epidemiology: The contribution of participatory research to epidemiology. Emerg Themes Epidemiol 2017;14:2 [FREE Full text] [CrossRef] [Medline]
- Koppeschaar CE, Colizza V, Guerrisi C, Turbelin C, Duggan J, Edmunds WJ, et al. Influenzanet: Citizens among 10 countries collaborating to monitor influenza in Europe. JMIR Public Health Surveill 2017 Sep 19;3(3):e66 [FREE Full text] [CrossRef] [Medline]
- Smolinski MS, Crawley AW, Olsen JM, Jayaraman T, Libel M. Participatory disease surveillance: Engaging communities directly in reporting, monitoring, and responding to health threats. JMIR Public Health Surveill 2017 Oct 11;3(4):e62 [FREE Full text] [CrossRef] [Medline]
- Boulos MN, Wheeler S, Tavares C, Jones R. How smartphones are changing the face of mobile and participatory healthcare: An overview, with example from eCAALYX. Biomed Eng Online 2011 Apr 05;10:24 [FREE Full text] [CrossRef] [Medline]
- Leal Neto OB, Albuquerque CM, Albuquerque JO, Barbosa CS. The schisto track: A system for gathering and monitoring epidemiological surveys by connecting geographical information systems in real time. JMIR Mhealth Uhealth 2014 Mar 10;2(1):e10 [FREE Full text] [CrossRef] [Medline]
- Leal Neto OB, Albuquerque J, de Souza WV, Cesse E, Cruz OG. Disruptive innovations and transformations in public health in the digital age [Article in Portuguese]. Cad Saude Publica 2017 Nov 21;33(11):e00005717 [FREE Full text] [CrossRef] [Medline]
- Leal Neto OB, Albuquerque J, Cruz OG, Cesse E, de Souza WV. Optimistic perspectives for the health of the future [Article in Portuguese]. Cad Saude Publica 2017 Nov 21;33(11):e00164017 [FREE Full text] [CrossRef] [Medline]
- Oliveira AD, Prats C, Espasa M, Zarzuela Serrat F, Montañola Sales C, Silgado A, et al. The malaria system microapp: A new, mobile device-based tool for malaria diagnosis. JMIR Res Protoc 2017 Apr 25;6(4):e70 [FREE Full text] [CrossRef] [Medline]
- Olson D, Lamb M, Lopez MR, Colborn K, Paniagua-Avila A, Zacarias A, et al. Performance of a mobile phone app-based participatory syndromic surveillance system for acute febrile illness and acute gastroenteritis in rural Guatemala. J Med Internet Res 2017 Nov 09;19(11):e368 [FREE Full text] [CrossRef] [Medline]
- Fujibayashi K, Takahashi H, Tanei M, Uehara Y, Yokokawa H, Naito T. A new influenza-tracking smartphone app (Flu-Report) based on a self-administered questionnaire: Cross-sectional study. JMIR Mhealth Uhealth 2018 Jun 06;6(6):e136 [FREE Full text] [CrossRef] [Medline]
- Baltrusaitis K, Santillana M, Crawley AW, Chunara R, Smolinski M, Brownstein JS. Determinants of participants' follow-up and characterization of representativeness in Flu Near You, a participatory disease surveillance system. JMIR Public Health Surveill 2017 Apr 07;3(2):e18 [FREE Full text] [CrossRef] [Medline]
- Swain C, Sawicki S, Addison D, Katz B, Piersanti K, Baim-Lance A, et al. An end-user participatory approach to collaboratively refine HIV care data, the New York State experience. AIDS Behav 2019 Jan;23(Suppl 1):83-93. [CrossRef] [Medline]
- Brownstein JS, Chu S, Marathe A, Marathe MV, Nguyen AT, Paolotti D, et al. Combining participatory influenza surveillance with modeling and forecasting: Three alternative approaches. JMIR Public Health Surveill 2017 Nov 01;3(4):e83 [FREE Full text] [CrossRef] [Medline]
- Smolinski MS, Crawley AW, Baltrusaitis K, Chunara R, Olsen JM, Wójcik O, et al. Flu Near You: Crowdsourced symptom reporting spanning 2 influenza seasons. Am J Public Health 2015 Oct;105(10):2124-2130. [CrossRef] [Medline]
- Chunara R, Goldstein E, Patterson-Lomba O, Brownstein JS. Estimating influenza attack rates in the United States using a participatory cohort. Sci Rep 2015 Apr 02;5:9540 [FREE Full text] [CrossRef] [Medline]
- Santillana M, Nguyen AT, Dredze M, Paul MJ, Nsoesie EO, Brownstein JS. Combining search, social media, and traditional data sources to improve influenza surveillance. PLoS Comput Biol 2015 Oct;11(10):e1004513 [FREE Full text] [CrossRef] [Medline]
- Leal Neto OB, Dimech GS, Libel M, de Souza WV, Cesse E, Smolinski M, et al. Saúde na Copa: The world's first application of participatory surveillance for a mass gathering at FIFA World Cup 2014, Brazil. JMIR Public Health Surveill 2017 May 04;3(2):e26 [FREE Full text] [CrossRef] [Medline]
- Cantarelli P, Debin M, Turbelin C, Poletto C, Blanchon T, Falchi A, et al. The representativeness of a European multi-center network for influenza-like-illness participatory surveillance. BMC Public Health 2014 Sep 20;14:984 [FREE Full text] [CrossRef] [Medline]
- Pini A, Merk H, Carnahan A, Galanis I, van Straten E, Danis K, et al. High added value of a population-based participatory surveillance system for community acute gastrointestinal, respiratory and influenza-like illnesses in Sweden, 2013-2014 using the Web. Epidemiol Infect 2017 Apr;145(6):1193-1202 [FREE Full text] [CrossRef] [Medline]
- Queenan K, Mangesho P, Ole-Neselle M, Karimuribo E, Rweyemamu M, Kock R, et al. Using local language syndromic terminology in participatory epidemiology: Lessons for One Health practitioners among the Maasai of Ngorongoro, Tanzania. Prev Vet Med 2017 Apr 01;139(Pt A):42-49. [CrossRef] [Medline]
- Gopichandran V. Community-based participatory epidemiology in tribal areas in India. J Postgrad Med 2017;63(2):79-80 [FREE Full text] [CrossRef] [Medline]
- Karimuribo ED, Mutagahywa E, Sindato C, Mboera L, Mwabukusi M, Kariuki Njenga M, et al. A smartphone app (AfyaData) for innovative one health disease surveillance from community to national levels in Africa: Intervention in disease surveillance. JMIR Public Health Surveill 2017 Dec 18;3(4):e94 [FREE Full text] [CrossRef] [Medline]
- Meyers D, Ozonoff AL, Baruwal A, Pande S, Harsha A, Sharma R, et al. Combining healthcare-based and participatory approaches to surveillance: Trends in diarrheal and respiratory conditions collected by a mobile phone system by community health workers in rural Nepal. PLoS One 2016;11(4):e0152738 [FREE Full text] [CrossRef] [Medline]
- Perrotta D, Bella A, Rizzo C, Paolotti D. Participatory online surveillance as a supplementary tool to sentinel doctors for influenza-like illness surveillance in Italy. PLoS One 2017;12(1):e0169801 [FREE Full text] [CrossRef] [Medline]
- Quade P, Nsoesie EO. A platform for crowdsourced foodborne illness surveillance: Description of users and reports. JMIR Public Health Surveill 2017 Jul 05;3(3):e42 [FREE Full text] [CrossRef] [Medline]
- Dalton C, Carlson S, Butler M, Cassano D, Clarke S, Fejsa J, et al. Insights from Flutracking: Thirteen tips to growing a Web-based participatory surveillance system. JMIR Public Health Surveill 2017 Aug 17;3(3):e48 [FREE Full text] [CrossRef] [Medline]
- Memish ZA, Stephens GM, Steffen R, Ahmed QA. Emergence of medicine for mass gatherings: Lessons from the Hajj. Lancet Infect Dis 2012 Jan;12(1):56-65. [CrossRef] [Medline]
- Rodriguez-Valero N, Borobia AM, Lago M, Sánchez-Seco MP, de Ory F, Vázquez A, et al. Zika virus screening among Spanish team members after 2016 Rio de Janeiro, Brazil, Olympic Games. Emerg Infect Dis 2017 Aug;23(8):1426-1428 [FREE Full text] [CrossRef] [Medline]
- Smolinski MS, Crawley AW, Olsen JM. Finding outbreaks faster. Health Secur 2017;15(2):215-220 [FREE Full text] [CrossRef] [Medline]
- GitHub. Guardiões da Saúde URL: https://github.com/Epitrack/guardioes-da-saude [accessed 2020-01-05]
- Guardians of Health. URL: https://www.guardioesdasaude.org/ [accessed 2020-02-05]
- Hall-Geisler K. TechCrunch. 2016 Oct 11. Waze and Esri make app-to-infrastructure possible URL: https://techcrunch.com/2016/10/11/waze-and-ezri-make-app-to-infrastructure-possible [accessed 2020-02-18]
- Seabrook HJ, Stromer JN, Shevkenek C, Bharwani A, de Grood J, Ghali WA. Medical applications: A database and characterization of apps in Apple iOS and Android platforms. BMC Res Notes 2014 Aug 27;7:573 [FREE Full text] [CrossRef] [Medline]
- Götz FM, Stieger S, Reips U. Users of the main smartphone operating systems (iOS, Android) differ only little in personality. PLoS One 2017;12(5):e0176921 [FREE Full text] [CrossRef] [Medline]
- Dehling T, Gao F, Schneider S, Sunyaev A. Exploring the far side of mobile health: Information security and privacy of mobile health apps on iOS and Android. JMIR Mhealth Uhealth 2015 Jan 19;3(1):e8 [FREE Full text] [CrossRef] [Medline]
- Leal Neto OB, Loyo R, Albuquerque J, Perazzo J, Barbosa V, Barbosa CS. Using mobile technology to conduct epidemiological investigations. Rev Soc Bras Med Trop 2015;48(1):105-107 [FREE Full text] [CrossRef] [Medline]
- Statista. Hamburg, Germany: Statista Research Department; 2020 Jan 07. Market share of mobile operating systems in Brazil from January 2012 to December 2018 URL: https://www.statista.com/statistics/262167/market-share-held-by-mobile- operating-systems-in-brazil/ [accessed 2020-02-18]
- Klotzbach C. Flurry. 2016 May 12. Enter the matrix: App retention and engagement URL: http://flurrymobile.tumblr.com/post/144245637325/appmatrix [accessed 2020-02-18]
- Lu FS, Hou S, Baltrusaitis K, Shah M, Leskovec J, Sosic R, et al. Accurate influenza monitoring and forecasting using novel internet data streams: A case study in the Boston metropolis. JMIR Public Health Surveill 2018 Jan 09;4(1):e4 [FREE Full text] [CrossRef] [Medline]
- Pagliari C, Vijaykumar S. Digital participatory surveillance and the Zika crisis: Opportunities and caveats. PLoS Negl Trop Dis 2016 Jun;10(6):e0004795 [FREE Full text] [CrossRef] [Medline]
- Salathé M. Digital epidemiology: What is it, and where is it going? Life Sci Soc Policy 2018 Jan 04;14(1):1 [FREE Full text] [CrossRef] [Medline]
- Patterson-Lomba O, Van Noort S, Cowling B, Wallinga J, Gomes G, Lipsitch M, et al. Utilizing syndromic surveillance data for estimating levels of influenza circulation. Am J Epidemiol 2014 Jun 01;179(11):1394-1401 [FREE Full text] [CrossRef] [Medline]
- Lwin MO, Jayasundar K, Sheldenkar A, Wijayamuni R, Wimalaratne P, Ernst KC, et al. Lessons from the implementation of Mo-Buzz, a mobile pandemic surveillance system for dengue. JMIR Public Health Surveill 2017 Oct 02;3(4):e65 [FREE Full text] [CrossRef] [Medline]
- Paolotti D, Carnahan A, Colizza V, Eames K, Edmunds J, Gomes G, et al. Web-based participatory surveillance of infectious diseases: The Influenzanet participatory surveillance experience. Clin Microbiol Infect 2014 Jan;20(1):17-21 [FREE Full text] [CrossRef] [Medline]
- Zimmerman L, Lounsbury DW, Rosen CS, Kimerling R, Trafton JA, Lindley SE. Participatory system dynamics modeling: Increasing stakeholder engagement and precision to improve implementation planning in systems. Adm Policy Ment Health 2016 Nov;43(6):834-849. [CrossRef] [Medline]
- Verdugo C, El Masry I, Makonnen Y, Hannah H, Unger F, Soliman M, et al. Sensitivity and specificity estimation for the clinical diagnosis of highly pathogenic avian influenza in the Egyptian participatory disease surveillance program. Avian Dis 2016 Dec;60(4):805-809. [CrossRef] [Medline]
- Kecojevic A, Basch C, Basch C, Kernan W. Pre-exposure prophylaxis YouTube videos: Content evaluation. JMIR Public Health Surveill 2018 Feb 16;4(1):e19 [FREE Full text] [CrossRef] [Medline]
- Hartley D, Nelson N, Arthur R, Barboza P, Collier N, Lightfoot N, et al. An overview of internet biosurveillance. Clin Microbiol Infect 2013 Nov;19(11):1006-1013 [FREE Full text] [CrossRef] [Medline]
|API: application programming interface|
|loess: locally estimated scatterplot smoothing|
|REST API: representational state transfer technology application programming interface|
Edited by T Sanchez; submitted 03.09.19; peer-reviewed by J Luiz de Lima Filho, A Nguyen, K Baltrusaitis, T Yano; comments to author 28.10.19; revised version received 06.12.19; accepted 27.01.20; published 07.04.20
©Onicio Batista Leal Neto, Oswaldo Cruz, Jones Albuquerque, Mariana Nacarato de Sousa, Mark Smolinski, Eduarda Ângela Pessoa Cesse, Marlo Libel, Wayner Vieira de Souza. Originally published in JMIR Public Health and Surveillance (http://publichealth.jmir.org), 07.04.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on http://publichealth.jmir.org, as well as this copyright and license information must be included.