This paper is in the following e-collection/theme issue:

Participatory Epidemiology and Surveillance (119) Crowdsourcing and Mechanical Turks (98) Infectious Diseases (non-STD/STI) (1575) Influenza and Influenza-Like-Illness (ILI) Surveillance (217) Surveillance Systems (462) mHealth for Data Collection and Research (955)

Published on in Vol 3, No 4 (2017): Oct-Dec

Will Participatory Syndromic Surveillance Work in Latin America? Piloting a Mobile Approach to Crowdsource Influenza-Like Illness Data in Guatemala

Will Participatory Syndromic Surveillance Work in Latin America? Piloting a Mobile Approach to Crowdsource Influenza-Like Illness Data in Guatemala

Will Participatory Syndromic Surveillance Work in Latin America? Piloting a Mobile Approach to Crowdsource Influenza-Like Illness Data in Guatemala

Authors of this article:

José Tomás Prieto1, 2 Author Orcid Image ;   Jorge H Jara1 Author Orcid Image ;   Juan Pablo Alvis1 Author Orcid Image ;   Luis R Furlan3 Author Orcid Image ;   Christian Travis Murray1 Author Orcid Image ;   Judith Garcia4 Author Orcid Image ;   Pierre-Jean Benghozi2 Author Orcid Image ;   Susan Cornelia Kaydos-Daniels5 Author Orcid Image
Article Authors Cited by (12) Tweetations (9) Metrics

    Original Paper

    1Center for Health Studies, Universidad del Valle de Guatemala, Guatemala City, Guatemala

    2Centre de Recherche en Gestion, Centre National de la Recherche Scientifique, Ecole Polytechnique, Palaiseau, France

    3Center for Studies in Applied Informatics, Universidad del Valle de Guatemala, Guatemala City, Guatemala

    4Ministry of Health, Guatemala City, Guatemala

    5Influenza Program, US Centers for Disease Control and Prevention, Guatemala City, Guatemala

    Corresponding Author:

    José Tomás Prieto, BEng, MSc, PhD

    Center for Health Studies, Universidad del Valle de Guatemala

    18 Av. 11-95, Zona 15, Vista Hermosa III

    Guatemala City, 01015

    Guatemala

    Phone: +1 4044216455

    Email: josetomasprieto@gmail.com


    Background: In many Latin American countries, official influenza reports are neither timely nor complete, and surveillance of influenza-like illness (ILI) remains thin in consistency and precision. Public participation with mobile technology may offer new ways of identifying nonmedically attended cases and reduce reporting delays, but no published studies to date have assessed the viability of ILI surveillance with mobile tools in Latin America. We implemented and assessed an ILI-tailored mobile health (mHealth) participatory reporting system.

    Objective: The objectives of this study were to evaluate the quality and characteristics of electronically collected data, the user acceptability of the symptom reporting platform, and the costs of running the system and of identifying ILI cases, and to use the collected data to characterize cases of reported ILI.

    Methods: We recruited the heads of 189 households comprising 584 persons during randomly selected home visits in Guatemala. From August 2016 to March 2017, participants used text messages or an app to report symptoms of ILI at home, the ages of the ILI cases, if medical attention was sought, and if medicines were bought in pharmacies. We sent weekly reminders to participants and compensated those who sent reports with phone credit. We assessed the simplicity, flexibility, acceptability, stability, timeliness, and data quality of the system.

    Results: Nearly half of the participants (47.1%, 89/189) sent one or more reports. We received 468 reports, 83.5% (391/468) via text message and 16.4% (77/468) via app. Nine-tenths of the reports (93.6%, 438/468) were received within 48 hours of the transmission of reminders. Over a quarter of the reports (26.5%, 124/468) indicated that at least someone at home had ILI symptoms. We identified 202 ILI cases and collected age information from almost three-fifths (58.4%, 118/202): 20 were aged between 0 and 5 years, 95 were aged between 6 and 64 years, and three were aged 65 years or older. Medications were purchased from pharmacies, without medical consultation, in 33.1% (41/124) of reported cases. Medical attention was sought in 27.4% (34/124) of reported cases. The cost of identifying an ILI case was US $6.00. We found a positive correlation (Pearson correlation coefficient=.8) between reported ILI and official surveillance data for noninfluenza viruses from weeks 41 (2016) to 13 (2017).

    Conclusions: Our system has the potential to serve as a practical complement to respiratory virus surveillance in Guatemala. Its strongest attributes are simplicity, flexibility, and timeliness. The biggest challenge was low enrollment caused by people’s fear of victimization and lack of phone credit. Authorities in Central America could test similar methods to improve the timeliness, and extend the breadth, of disease surveillance. It may allow them to rapidly detect localized or unusual circulation of acute respiratory illness and trigger appropriate public health actions.

    JMIR Public Health Surveill 2017;3(4):e87

    doi:10.2196/publichealth.8610

    Keywords

    crowdsourcing; human flu; influenza; grippe; mHealth; texting; mobile apps; short message service; text message; developing countries 


    Influenza Surveillance in Latin America

    The routine collection, analysis, and reporting of disease and injury data is the foundation of public health: it provides health officials with current information to ground public health decisions and action [1,2]. Despite efforts to improve disease surveillance and response, developing countries still have difficulty in accurately identifying, diagnosing, and reporting infectious diseases [3]. In Latin America, influenza and other respiratory virus surveillance lacks timeliness and is nonspecific, particularly in remote areas [4-7]. The recommendations that were proposed more than a decade ago to advance surveillance [7]—improving the quality of available data, enhancing communication of the information obtained by surveillance activities to stakeholders, and increasing awareness of health authorities of influenza impact in public health—have yet to be fully implemented in the region [5].

    Guatemala is illustrative of this challenge. Disease surveillance activities are constrained by scarcity of resources, personnel, and infrastructure [8,9]. Seasonal influenza information is incomplete and delayed because reports are publicly available 2 to 3 weeks after the emergence of medically attended cases from which samples are collected. Low usage of government health facilities conducting sentinel surveillance of influenza makes it difficult to accurately estimate the geographic distribution of influenza in the country [8]. The effect of the data shortcomings is that authorities and providers are forced to make decisions, such as defining the timing of effective vaccination campaigns, using inadequate information on a seasonal disease. Countries in the region facing the same challenges are in need of effective tools to improve the quality and timeliness of surveillance data.

    The Promise of Culturally Relevant Participatory Surveillance

    Participatory surveillance strategies, by which individuals report symptoms voluntarily, have the potential to serve as a viable complement to existing outpatient, hospital-based, and laboratory surveillance systems because they can be flexible, sensitive, and accurate [10,11]. Studies suggest that electronic participatory surveillance may mitigate delays in reporting, improve timeliness of surveillance, extend surveillance to unmonitored locations, reduce costs, increase the population size under surveillance, and characterize cases of nonmedically attended illnesses [10,12-19]. Although attention has mainly been focused on using data from Internet posts and search queries for influenza surveillance [18], other approaches to participatory surveillance of influenza, such as Flu Near You [10] (and its extension Salud Boricua [20] in Puerto Rico), InfluenzaNet [21,22], or the Web-based Reporta [23] in Mexico, have generated relevant information by collecting data from users who periodically report symptoms; the data have been used to signal the beginning of yearly influenza epidemics, allowing an appropriate mobilization of communities to seek vaccines and a targeted promotion of nonpharmaceutical interventions such as avoiding close contact or covering the mouth and nose when coughing or sneezing [21].

    An electronic bottom-up surveillance system may offer new ways of strengthening situational awareness by engaging citizens in countries such as Guatemala. Implementing functional participatory surveillance systems, however, requires designing systems that are appropriate and scalable, and developing local strategies for encouraging participation [11,24]. In Latin America, the unsuitability of Google Flu Trends to accurately predict influenza outbreaks across countries has been attributed to low Internet penetration rates and to heterogeneous usage of search engines among non-English speaking populations [24]. Alternatively, the ubiquity of feature phones—a midway point between smartphones and basic mobile phones [25], which are only capable of voice calling and text messaging—coupled with the increasing penetration of smartphones provides culturally relevant infrastructure opportunities to capture health information with the use of apps and short message service (SMS) (or text messages) [26,27]; it is almost four times more likely to be a mobile phone user than an Internet user in Central America (34 Internet users per 100 people [28] vs 130 mobile subscriptions [29] per 100 people). No published studies to date have assessed the viability of ILI surveillance with mobile-based tools in the region.

    The Study in Guatemala

    We implemented and assessed a participatory, bimodal, mobile phone–based surveillance system for ILI symptoms, the first in the Latin American region. The mobile health (mHealth) system was purposely developed to make use of popular mobile communication tools—SMS and Android apps—in Guatemala. SMS provides a cross-platform mobile communication channel that is widely used in the country, where mobile penetration rates exceed 100% [29]. Android smartphones dominate Central and South American markets with approximately 80% of the market share [30]; in Guatemala, the leading telecom—with 54.5% of active mobile lines—reported that the four most used smartphones in 2016 ran on the Android operating system [28]. We conducted a study between August 2016 and March 2017 in San Marcos, Guatemala, to assess the relevance, performance, and potential of the ILI-tailored reporting system. Specific aims were to evaluate the quality and characteristics of the collected data, the user acceptability of the platform, and the costs of running the system and of identifying ILI cases and to use the collected data to characterize cases of reported ILI in the study population.


    The Bimodal Data Capturing System

    We created a system capable of capturing ILI reports from two sources, SMS and the smartphone app “Mi Gripe” (“My Flu” in Spanish) (see Figure 1 for screenshots). Our ILI case definition was as follows: measured fever and cough or sore throat, with onset within the last 7 days.

    We used a Dell Laptop running on Microsoft Windows 7 Professional with an Intel Core Duo processor and 4-GB RAM, a Huawei E173 GSM Modem connected to the laptop, and a subscriber identity module (SIM) card compatible with the GSM Modem. We used the open-source program FrontlineSMS [31] to import contact datasets and to schedule the delivery of weekly reminders. We configured the software to send automatic follow-up questions to participants based on keywords contained in incoming SMSs. The logic for the SMS exchange was as follows: at the beginning of each week, the platform sent the question: “How many people at home had fever with cough or sore throat in the last 7 days? Send 0 if none. Respond with the number if there were any.” If the answer was 0, the system sent a “Thank you” message. If the answer was a number between 1 and 15, the system sent the next question: “Did the people who had fever with cough or sore throat in the last 7 days seek medical attention (excluding traditional medicine and pharmacies)? Send 111 if yes, 222 if no.” If the answer contained either 111 or 222, the system sent the question: “Did the people who had fever and cough or sore throat in the last 7 days buy medicines at a pharmacy? Send 333 if yes, 444 if no.” If the answer contained 333 or 444, the following question was sent automatically: “What are the ages of the people who had fever with cough or sore throat? Send AGE followed by their ages.” If the system identified the keyword AGE in the text, a “Thank you” message was sent, marking the end of a weekly SMS report (see Multimedia Appendices 1 and 2).

    We also created the app “Mi Gripe” for Android smartphones. The app “Mi Gripe” allowed participants to complete a weekly report by answering a set of questions (the same ones we used in the SMS modality). A reminder popped up every week to remind users to complete the weekly report. Participants could not skip questions in the app because data entry validation required them to answer questions before moving to the next one. A message was displayed if the report for the week had been sent. Reports were sent if and when the mobile device found connectivity to the Internet.

    Location and Participants

    The study was conducted between August 2016 and March 2017 in southwestern Guatemala, close to the Mexican border and the Pacific Ocean coast, in the municipalities of San Marcos and San Pedro (Department of San Marcos).

    As part of a health utilization survey conducted in the coverage area of the San Marcos and San Pedro public health centers, we selected 1671 households using a cluster random sample strategy (paper forthcoming). We invited heads of Spanish-speaking households to participate in the study. The inclusion criteria for the study were as follows: households with at least one mobile phone and households where at least one person knew how to read and write in Spanish. Visits were organized as follows: clusters of houses in the coverage area of the health centers were first selected using a probabilistic cluster stratified sampling.

    ‎
    Figure 1. Two screenshots of the app “Mi Gripe” (left and middle) and screenshot of exchanges for short message service (SMS) users (right). Screenshot A: Translation: "Welcome to the Mi Gripe App. Please remember to send your weekly report." Screenshot B: Translation: “How many people at home had fever with cough or sore throat in the last 7 days? None." Screenshot C: Translation: “Did the people who had fever with cough or sore throat in the last 7 days seek medical attention, excluding traditional medicine and pharmacies? Send 111 if yes, 222 if no,” “Did the people who had fever and cough or sore throat in the last 7 days buy medicines at a pharmacy? Send 333 if yes, 444 if no,” “What are the ages of the people who had fever with cough or sore throat? Send AGE followed by their ages.”.
    View this figure

    The stratification was based on the location (urban or rural) of houses. Within each cluster, a random sample of households was selected to conduct a health utilization survey, which would help estimate the burden of influenza and other respiratory viruses in sentinel surveillance clinics, with each head of the household. At the end of the 1- to 2-hour survey, the field personnel invited the heads of households to participate in the study.

    Interested participants who signed an informed consent form were instructed to send reports once per week via SMS if they had a feature phone or via app if they had a smartphone, using their phone credit. The study’s field personnel installed “Mi Gripe” on smartphones of app participants and saved the reporting number under the contact name “Report symptoms” for SMS participants. Participants were credited with ~US $0.40 for each report sent at the end of each month (to cover the cost of sending 4 SMSs). Maximum compensation was set at ~US $1.60 per month, thus compensating no more than four reports per month.

    Coding of SMS Reports

    SMS participants were instructed to follow response rules described at the end of every text message, for example, “Send AGE followed by the ages” (details are provided in Multimedia Appendices 1 and 2). At the design stage, we anticipated that participant reports would not systematically comply with the response rules. We therefore prospectively included variants of possible responses in the automatic response system created in FrontlineSMS to optimize interactions with participants. At the end of the project, investigators manually reviewed all answers and recoded them when necessary to have standardized answers for each question, to calculate response times, and to simplify data aggregation and analysis.

    Analysis

    We used descriptive statistics to characterize households and electronic reports and the t test to compare means. A significance level of .05 was prespecified for all comparisons. The Pan American Health Organization (PAHO) publishes in the Health Intelligence Platform [32] the weekly number of laboratory-confirmed cases of respiratory viruses. In Guatemala, information is based on samples collected by sentinel surveillance sites: we calculated the Pearson correlation test to investigate the correlation between the number of electronically identified ILI cases in participants’ reports and the official number of reported, laboratory-confirmed cases of influenza and other respiratory viruses (respiratory syncytial virus [RSV], metapneumovirus, adenoviruses, and parainfluenza viruses) published by PAHO, for Guatemala. We hypothesized that there would be a weak to moderate positive correlation (.20≤ r ≤.59) between the number of electronically identified ILI cases and the number of reported, laboratory-confirmed cases of respiratory virus during the study period. Costs incurred for text messages, participant compensation, electricity, rental space, and personnel are presented in US dollars. We summarized age information of electronically identified ILI cases by age group of people at high risk for developing flu-related complications (children younger than 5 years and adults 65 years of age and older). Statistical analysis was conducted using STATA, version 13.0 for Macintosh, and Microsoft Excel for Mac 2011. On the basis of the Centers of Disease Control and Prevention (CDC) Updated Guidelines for Evaluating Public Health Surveillance Systems [33], we assessed, in Discussion, the simplicity, flexibility, acceptability, stability, timeliness, representativeness, and data quality of the system. All data generated or analyzed during this study are included in this published paper and its Multimedia Appendices. Further information from the study can be requested from the corresponding author on reasonable request.

    Ethics Approval

    This study was conducted in accordance with all applicable local regulatory requirements and the principles of the Declaration of Helsinki. The study was approved by the Ethics Committee of Universidad del Valle de Guatemala on July 5, 2016, and by the Guatemalan National Ethics Committee on August 5, 2016 (resolution 17-2016). Both committees provided ethical oversight during the study.


    Demographic Information

    Approximately one-fifth (20.41%, 341/1671) of visited households were not eligible because the head of household reported not having a mobile phone at home. The majority of eligible households (85.79%, 1141/1330) refused to participate because of security concerns or lack of credit at the time of enrollment. We recruited the heads of 189 eligible households (14.21%, 189/1330), comprising 584 people.

    Among participants, 84.1% (159/189) used feature phones, and 15.9% (30/189) used smartphones. During the home visits, 177 households provided demographic information. The mean age of people at participating homes was 28 years and ranged between 0 and 89 years. The mean age of head of households was 40 years (Table 1).

    Only 22.0% (39/177) of participants had access to a fixed telephone line at home. Households in the SMS and app modalities were similar: the t test revealed no significant differences between SMS and app households in terms of the number of people per home, the ages of people living at home, or the ages of the heads of households.

    SMS and App Reports

    From August 2016 to March 2017, we received 468 reports, a mean of 2.5 reports per participant household (Table 2).

    Table 1. Demographic information of participant households.
    IndicatorModalityTotal

    		SMSaApp
    Number of recruited households16029189
    Number of people per homeN=149N=28N=177

    		Mean (IQRb)3 (2-4)4 (3-5)3 (2-4)

    		Range1-71-61-7
    Age of people in householdN=149N=28N=177

    		Mean292728

    		Median (IQR)26 (15-42)23 (15-40)25 (25-42)

    		Minimum-maximum0-890-730-89
    Age of head of householdN=151N=26N=177

    		Mean404040.3

    		Median (IQR)39 (28-51)39 (25-50)39 (28-50)

    		Minimum-maximum18-8320-6818-83
    Percentage of homes withN=151N=26N=177

    		Electricity, n (%)148 (98.0)26 (100)98.3 (17.4)

    		Running water, n (%)150 (99.3)25 (96)175 (98.9)

    		Fixed telephone line, n (%)33 (21.8)6 (23)39 (22.0)
    Education of head of householdN=141N=25N=166

    		No studies, n (%)4 (2.8)1 (4)5 (3.0)

    		Some primary school, n (%)42 (30.0)5 (20)47 (28.3)

    		Some secondary school, n (%)26 (18.4)4 (16)30 (18.1)

    		Some university, n (%)69 (48.9)15 (60)84 (50.6)

    aSMS: short message service.

    bIQR: interquartile range.

    Table 2. Reporting from August 2016 to March 2017.
    IndicatorSMSaAppTotal
    Reports per user



    		Mean2.42.62.5

    		Minimum-maximum0-250-250-25
    Participants who



    		Sent no reports, n (%)85 (53.1)15 (51.7)100 (52.9)

    		Sent between 1 and 5 reports, n (%)56 (35.0)11 (37.9)67 (35.5)

    		Sent between 6 and 15 reports, n (%)9 (5.6)1 (3.5)10 (5.3)

    		Sent 16 or more reports, n (%)10 (6.3)2 (6.9)12 (6.3)
    Age group of ILIbcases



    		0-5 years13720

    		6-64 years732295

    		>64 years033

    aSMS: short service message.

    bILI: influenza-like illness.

    More than four-fifths (83.5%, 391/468) of reports were received via SMS and the remainder (16.4%, 77/468) via the app “Mi Gripe.” Almost nine-tenths (88.0%, 412/468) of reports were received within 24 hours and 93.6% (438/468) within 48 hours of transmission of weekly reminders. Nearly half (47.1%, 89/189) of participants sent one or more reports. Moreover, 12 participants (6.3%, 12/189) sent 16 or more reports during the study.

    Nearly three quarters of reports (73.5%, 344/468) indicated that no one at home developed ILI symptoms during the project period. Over a quarter of reports indicated that someone at home developed ILI symptoms (26.5%, 124/468). The 124 reports indicated that 202 individuals presented ILI symptoms during the study (see Figure 2 for weekly details about the number of reported ILI episodes to the electronic platform and the number of official laboratory-confirmed cases of respiratory viruses in Guatemala).

    We received age data for 118 of those 202 cases: the majority (80.5%, 95/118) were aged between 6 and 64 years, 16.9% (20/118) were aged between 0 and 5 years, and three (2.6%, 3/118) were 75 years old. Most ILI cases for which we had age data (89.0%, 105/118) occurred among family members rather than the respondent: in 105 cases, the ages of the ILI cases did not correspond to the ages of the heads of households.

    According to PAHO data [32], the number of reported, laboratory-confirmed cases of influenza in Guatemala started to increase in epidemiological weeks 6 and 7 of 2016 and started to decrease in week 12, with the end of the influenza season around week 17 of 2016. The number of reported, laboratory-confirmed cases of RSV, metapneumovirus, adenovirus, and parainfluenza started to increase between epidemiological weeks 20 and 23 of 2016 and started to decrease between weeks 40 to 41 of 2016, with the end of the season around week 1 of 2017.

    We found a positive correlation (Pearson correlation coefficient=.8) between the number of ILI cases reported to our system and the number of reported, laboratory-confirmed cases of noninfluenza respiratory viruses (RSV, metapneumovirus, adenovirus, and parainfluenza virus) in Guatemala, published by PAHO from epidemiological week 41 in 2016 to 13 in 2017. The correlation was weaker (Pearson correlation coefficient=.6) when we combined the number of reported, laboratory-confirmed cases of influenza and other respiratory virus cases (Table 3).

    We found a negative correlation (r=−.3) between the number of ILI cases reported to our system and the number of reported, laboratory-confirmed cases of influenza published by PAHO. No correlation was found between positivity for electronically identified ILI (defined as the number of electronically identified cases of ILI divided by the total number of household members involved in weekly reports) and positivity for laboratory-confirmed respiratory viruses in Guatemala in 2016 and 2017 (Multimedia Appendix 3).

    ‎
    Figure 2. Electronically-identified influenza-like illness (ILI) in San Marcos and official respiratory virus surveillance data in Guatemala, 2016 and 2017. *Weekly reminders were not sent to participants from weeks 38 to 40, when the project was interrupted. In week 10 of 2017, the GSM modem was accidentally disconnected from the short message service (SMS) platform and the scheduled reminders were not sent. **Official respiratory virus surveillance activities are normally stopped the last 2 weeks of every year. ***Noninfluenza respiratory viruses for the considered period include the following: respiratory syncytial virus, metapneumovirus, adenovirus, and parainfluenza virus.
    View this figure
    Table 3. Pearson correlation between electronically identified influenza-like illness (ILI) in San Marcos and San Pedro and respiratory virus laboratory data published by the Pan American Health Organization for Guatemala.
    VariablesPearson correlation, r

    		From epidemiological week 41 in 2016 to 13 in 2017aFrom epidemiological week 41 in 2016 to 13 in 2017 (excluding 10, 51, and 52)b
    Influenza and electronically identified ILIc−.412−.342
    Noninfluenza respiratory viruses and electronically identified ILI.656.802
    Influenza and other respiratory viruses and electronically identified ILI.373.623

    aThe project started in epidemiological week 35 of 2016 and was temporarily interrupted from epidemiological weeks 38 to 40. We considered data from week 41 in 2016 to 13 in 2017 to conduct the correlation analysis.

    bWe excluded weeks 51 and 52 of 2016 because official respiratory virus surveillance activities are stopped during the last 2 weeks of the year, and no official surveillance data are published. We excluded week 10 of 2017 because the GSM modem was accidentally disconnected from the SMS platform and the scheduled reminders were not sent.

    cILI: influenza-like illness.

    Costs

    Running the ILI surveillance system for 7 months cost approximately US $1212.58: one mHealth operator monitored the electronic platforms 3 hours per week during the study, amounting to US $738.21 in personnel; the system sent 5147 text messages, amounting to US $275.24; we reimbursed participants who sent a complete report either via SMS or the app, amounting to US $158.02; and the laptop and the server receiving reports were turned on 24 hours a day for 212 days, amounting to US $5.77 in electricity and US $35.33 in rental space. The cost of identifying one new ILI case in electronic reports was US $6 (details are provided in Multimedia Appendix 4).


    Summary

    The results show the ability of the developed system to capture ILI data that could be used to improve the timeliness and expand the breadth of syndromic surveillance activities. The data collected could be used to rapidly detect localized or unusual circulation of acute respiratory illness. The strategy could be further explored to determine the start of epidemic activity of respiratory viruses each year. This would allow authorities to trigger appropriate public health action such as collecting samples outside of normal surveillance, increasing communication to the public about the need to vaccinate target groups, and promoting nonpharmaceutical interventions to prevent transmission of viruses. The pilot allowed us to monitor the performance of the participatory surveillance system and assess some of its attributes, particularly its simplicity, flexibility, acceptability, stability, timeliness, representativeness, and data quality.

    Simplicity, Flexibility, and Timeliness

    The system was designed to maximize ease of operation. By using autonomous processes (SMS and app-driven), we minimized time spent on maintaining and monitoring the system. The reporting platform was sufficiently stable: it remained fully functional for 7 months, except for 1 week in March 2017, when the GSM modem was accidentally disconnected from the SMS platform. The reporting platform was accessible through the most popular mobile communication methods in the region. From the user’s perspective, it was designed to be intuitive to maximize acceptability. Nearly half of the participants (47.1%, 89/189) sent at least one report. In total, we received 468 reports through SMS and the app. The rate of long-term, consistently reporting participants in our project (those who sent 16 reports or more) was lower than in a similar project in the United States [10] (6.3% vs 10.4%).

    An important characteristic of the system was its timeliness: it generated data more rapidly than official surveillance methods. Almost all (93.6%, 438/468) ILI reports were received within 48 hours of transmission of weekly reminders, whereas the average time between symptoms onset and laboratory tests in sentinel surveillance sites in Guatemala was 161 hours (almost 7 days) in 2014 (data provided by the Department of Epidemiology of the Guatemalan Ministry of Health). The PAHO surveillance data that we used to calculate correlations took 2 to 3 weeks to become public during the study period. In a scaled-up implementation, this approach would allow the enactment of immediate actions while waiting for laboratory results.

    Results were derived from reports sent by the heads of participant households. The coding of SMS reports revealed that, even when instructed to do so, the reporters did not consistently comply with the predefined response rules. During the study, we prospectively adjusted keyword-based responses to gradually improve interactions with participants and increase the flexibility of the system. At the end of the project, we recoded the remaining uncategorized reports. Due to the informational benefits of each SMS—rule-compliant or not—associated efforts in developing tools that assist in the interpretation of their meaning are justified. A useful participatory electronic system must be able to capture structured and unstructured data, as well as expected and unexpected data, and transform them into relevant high-quality information in a timely manner [34]. A future version of the system could use this knowledge to train machine-learning algorithms that would automatically code incoming reports and self-increase flexibility. This would enable the generation of real-time data graphs and statistics for epidemiologists and public health experts. With modest development and configuration efforts, the approach may also make it possible to monitor other syndromes and contribute to the identification of febrile disease outbreaks in the region, such as dengue, chikungunya, or zika. It may also contribute to improved response to disease outbreaks by providing near real-time syndromic data and rapidly detecting localized and unusual incidence of specific symptoms [35-37]. Such information could prompt public health authorities to focus resources on actively sampling cases to determine the etiology of an ILI cluster.

    Representativeness, Data Quality, and New Information

    The pilot surveillance system was not designed to be representative; it was a proof-of-concept for participatory surveillance in Guatemala. Reports submitted during the study may in fact not be representative of all Guatemalan households either in terms of the incidence of ILI or of willingness to participate in the participatory system. One-seventh (14.21%, 189/1330) of eligible households accepted to participate in the study. The low uptake is consistent with that of similar projects [10] but might be explained by three different factors. First, participants were invited to the project after answering a 1- to 2-hour-long survey. Individuals were fatigued by the lengthy questionnaire and only occasionally wanted to hear about the project. Second, the area of San Marcos in Guatemala is particularly affected by gang violence. The heads of households often communicated their concerns about security to the field personnel when deciding to participate. Third, participants were told that they would be compensated for every report they would send during the study; however, many of them did not have phone credit at enrollment and were not interested in being reimbursed at the end of the month. A more effective strategy could have been to invite participants to the study before conducting the survey, give credit to interested participants before sending reminders, and let participants handle the installation of the app or the saving of the reporting number.

    Reports suggest that on 41 occasions (33.1% of reported ILI cases; 41/124), people at home self-medicated and only 27.4% (34/124) of participants who reported ILI symptoms sought medical attention (vs 35% for Flu Near You users [38]). On 50 occasions (40.3% of reported ILI cases; 50/124), medical attention was not sought, thus impeding the collection of samples and a laboratory confirmation of their diagnosis. The project generated information during epidemiological weeks 51 and 52 of 2016, when no official data were published for any respiratory virus in Guatemala [32]. This new information about health-seeking behavior during ILI episodes may be used to estimate the burden of the syndrome in the San Marcos department using recent modeling methods [39,40]; with additional representative data coming from more locations, it might also be used to estimate the burden of ILI in the country, which is likely substantial [41,42]. The approach presented here could contribute to strengthen disease surveillance activities in neighboring countries (Honduras, El Salvador, Nicaragua, and Belize) that have similar challenges in conducting influenza and other respiratory virus surveillance. As highlighted in a recent scientific commentary [43], crowdsourcing could make governments more agile in responding to public health problems. The strategy echoes the European Influenzanet in its ability to monitor ILI in the population, including individuals who do not seek medical attention [22].

    Retrospective credit compensation and weekly reminders were used to secure the periodic contributions of participants. The importance of weekly reminders was serendipitously demonstrated; from weeks 39 to 41 in 2016 and in week 10 in 2017, reminders were not sent to participants and the number of reports decreased dramatically (Figure 2). But even when weekly reminders were sent, results reveal that more than half (52.9%, 100/189) of participants did not send reports and almost one quarter (23.3%, 44/189) sent one or two reports only during the study. The finding suggests that new sustainable arrangements and incentives should be tested to motivate a continued participation. The feeding of weekly professional health information in exchange for reports and gamification methods, for example, could be effective ways of engaging citizens in the participatory system [44-46], especially in Latin American countries where access to public health centers is challenging and the demand for health information is expected to intensify [47]. Future systems should exploit lessons learned from related systems, such as Flutracking in Australia [48], to engage and motivate sustained participation. Some of these lessons seem relevant for the Latin-American context (eg, collect minimum answers and invite participants to report on household members [48]), but others (eg, use social media and website analytics or leverage organizational email invitations [48]) should be further explored to create a culturally appropriate and sustainable system in the region.

    Electronic Participatory ILI Surveillance Opportunities in the Region

    The collected data allowed us to identify 202 cases of ILI at home at a cost of less than US $6.00 per identified case. We did not find a strong correlation between the number of electronically identified ILI and the number of reported, laboratory-confirmed cases of influenza, probably because the 2016 flu season ended around May in Guatemala [32]; however, we found a strong positive correlation (r=.8) between the number of electronically identified ILI and the number of reported, laboratory-confirmed cases of noninfluenza respiratory viruses cases from epidemiological week 41 in 2016 to 13 in 2017 (Table 3; Figure 2), precisely during a period of increased circulation of noninfluenza respiratory viruses. The study shows the viability of capturing valuable citizen-sourced health data and seems more culturally appropriate than other recently studied alternatives [24,30] in the Latin-American realm. Our choice of reporting platforms, SMS and an Android app, maximizes the chances of having a culturally appropriate solution and minimizes additional software development efforts in a scale-up scenario for Central America. We plan to apply lessons learned and measure associations between self-reported ILI and laboratory-confirmed respiratory virus surveillance in a forthcoming, larger-scale crowdsourcing project.

    Acknowledgments

    This study was financed through the Cooperative Agreement 5U01GH001003-03 between the Centers for Disease Control and Prevention and Universidad del Valle de Guatemala. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

    The authors are grateful to Eduardo Azziz-Baumgartner for his comments about the manuscript. The authors are grateful to Antonio Paredes from the Epidemiology Department of the Guatemalan Ministry of Health and to Dr Elisa Barrios, epidemiologist in San Marcos, for their support during the presentation of the study protocol. The authors are grateful to Juan Carlos Romero from the Center for Health Studies of Universidad del Valle de Guatemala (UVG) for his contributions in the “Mi Gripe” app. The authors are also grateful to Julio Ayala and Marlon Castillo, computer science engineering students at UVG in 2016, for their assistance during participant enrolment in San Marcos. Special thanks for administrative and information technology (IT) support to Angelica Román, Victor Sicajau, Alexander Ramirez, Astri Alvarado, and Rafael Galvez from the Center for Health Studies.

    Authors' Contributions

    JTP devised the study, was the principal investigator, and wrote the first draft of this paper. SCK, JJ, and JPA contributed to the study design, interpretation of results, and revised the paper. PJB, LF, CM, and JG contributed to interpretation of results and revised the paper. JTP had full access to all data. All authors read and approved the final manuscript.

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    Processes for the app reporting platform.

    PNG File, 53KB
    ‎

    Multimedia Appendix 2

    Processes for the SMS reporting platform.

    PNG File, 67KB
    ‎

    Multimedia Appendix 3

    Positivity for electronically identified ILI (defined as the number of electronically identified cases of ILI divided by the total number of household members involved in weekly reports) and positivity for laboratory-confirmed respiratory viruses in Guatemala 2016 and 2017.

    PNG File, 171KB
    ‎

    Multimedia Appendix 4

    Dataset and main calculations.

    XLSX File (Microsoft Excel File), 660KB
    1. Centers for Disease Control (CDC). Comprehensive Plan for Epidemiologic Surveillance. Atlanta, GA: U.S. Department of Health and Human Services, CDC; Aug 1986.
    2. Cutts FT, Waldman RJ, Zoffman HM. Surveillance for the expanded programme on immunization. Bull World Health Organ 1993;71(5):633-639 [FREE Full text] [Medline]
    3. Apps.who.int. Switzerland: World Health Organization; 2012. Global Report for Research on Infectious Diseases of Poverty   URL: http://apps.who.int/iris/bitstream/10665/44850/1/9789241564489_eng.pdf [accessed 2017-11-01] [WebCite Cache]
    4. Butler D. Flu surveillance lacking. Nature 2012 Mar 28;483(7391):520-522. [CrossRef] [Medline]
    5. Bonvehí PE, Istúriz RE, Labarca JA, Rüttimann RW, Vidal EI, Vilar-Compte D. Influenza among adults in Latin America, current status, and future directions: a consensus statement. Rev Panam Salud Pública 2012;31(6):506-512. [CrossRef]
    6. Hoen AG, Keller M, Verma AD, Buckeridge DL, Brownstein JS. Electronic event-based surveillance for monitoring dengue, Latin America. Emerg Infect Dis 2012 Jul;18(7):1147-1150 [FREE Full text] [CrossRef] [Medline]
    7. Savy V. Regional perspectives on influenza surveillance in South America. Vaccine 2002 May 15;20 Suppl 2:S47-S49. [Medline]
    8. Lindblade KA, Johnson AJ, Arvelo W, Zhang X, Jordan HT, Reyes L, et al. Low usage of government healthcare facilities for acute respiratory infections in guatemala: implications for influenza surveillance. BMC Public Health 2011 Nov 24;11:885 [FREE Full text] [CrossRef] [Medline]
    9. Reyes L, Arvelo W, Estevez A, Gray J, Moir JC, Gordillo B, et al. Population-based surveillance for 2009 pandemic influenza A (H1N1) virus in Guatemala, 2009. Influenza Other Respir Viruses 2010 May 01;4(3):129-140 [FREE Full text] [CrossRef] [Medline]
    10. Smolinski MS, Crawley AW, Baltrusaitis K, Chunara R, Olsen JM, Wójcik O, et al. Flu Near You: crowdsourced symptom reporting spanning 2 influenza seasons. Am J Public Health 2015 Oct;105(10):2124-2130. [CrossRef] [Medline]
    11. Johansson M, Wojcik O, Chunara R, Smolinski M, Brownstein J. Participatory disease surveillance in Latin America. 2013 Presented at: Proceedings of the 22nd International Conference on World Wide Web: WWW '13 Companion; May 13-17, 2013; Rio de Janeiro, Brazil. [CrossRef]
    12. Polgreen PM, Chen Y, Pennock DM, Nelson FD. Using internet searches for influenza surveillance. Clin Infect Dis 2008 Dec 1;47(11):1443-1448 [FREE Full text] [CrossRef] [Medline]
    13. Mandl KD, Overhage JM, Wagner MM, Lober WB, Sebastiani P, Mostashari F, et al. Implementing syndromic surveillance: a practical guide informed by the early experience. J Am Med Inform Assoc 2004;11(2):141-150 [FREE Full text] [CrossRef] [Medline]
    14. Signorini A, Segre AM, Polgreen PM. The use of Twitter to track levels of disease activity and public concern in the U.S. during the influenza A H1N1 pandemic. PLoS One 2011;6(5):e19467 [FREE Full text] [CrossRef] [Medline]
    15. Eiji A, Sachiko M, Mizuki M. Twitter catches the flu: detecting influenza epidemics using Twitter. Stroudsburg, PA, USA: Association for Computational Linguistics; 2011 Presented at: Proceedings of the Conference on Empirical Methods in Natural Language Processing; July 27-31, 2011; Edinburgh, United Kingdom p. 1568-1576.
    16. Carneiro HA, Mylonakis E. Google trends: a web-based tool for real-time surveillance of disease outbreaks. Clin Infect Dis 2009 Nov 15;49(10):1557-1564 [FREE Full text] [CrossRef] [Medline]
    17. Choi H, Varian H. Predicting the present with Google Trends. Econ Rec 2012;88(s1):2-9. [CrossRef]
    18. Althouse BM, Scarpino SV, Meyers LA, Ayers JW, Bargsten M, Baumbach J, et al. Enhancing disease surveillance with novel data streams: challenges and opportunities. EPJ Data Sci 2015;4:pii: 17 [FREE Full text] [CrossRef] [Medline]
    19. Eysenbach G. Infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the Internet. J Med Internet Res 2009;11(1):e11 [FREE Full text] [CrossRef] [Medline]
    20. Saludboricua. 2015. Proyecto salud Boricua - salud en Puerto Rico 2015   URL: https://saludboricua.org/ [accessed 2017-09-10] [WebCite Cache]
    21. Paolotti D, Carnahan A, Colizza V, Eames K, Edmunds J, Gomes G, et al. Web-based participatory surveillance of infectious diseases: the Influenzanet participatory surveillance experience. Clin Microbiol Infect 2014 Jan;20(1):17-21 [FREE Full text] [CrossRef] [Medline]
    22. Guerrisi C, Turbelin C, Blanchon T, Hanslik T, Bonmarin I, Levy-Bruhl D, et al. Participatory syndromic surveillance of influenza in Europe. J Infect Dis 2016 Nov 14;214(suppl 4):S386-S392. [CrossRef]
    23. reporta.c3.org.mx. 2017. Reporta: Sistema Ciudadano de Monitoreo Epidemiológico   URL: http://reporta.c3.org.mx/ [WebCite Cache]
    24. Pollett S, Boscardin WJ, Azziz-Baumgartner E, Tinoco YO, Soto G, Romero C, et al. Evaluating Google Flu Trends in Latin America: important lessons for the next phase of digital disease detection. Clin Infect Dis 2017 Jan 01;64(1):34-41. [CrossRef] [Medline]
    25. Cnet. 2010. The 411: feature phones vs. smartphones   URL: https://www.cnet.com/news/the-411-feature-phones-vs-smartphones/ [accessed 2017-07-28] [WebCite Cache]
    26. Déglise C, Suggs LS, Odermatt P. SMS for disease control in developing countries: a systematic review of mobile health applications. J Telemed Telecare 2012 Jul;18(5):273-281. [CrossRef] [Medline]
    27. Dang LT, Vu NC, Vu TD, James SL, Katona P, Katona L, et al. Perceptions of the feasibility and practicalities of text messaging-based infectious disease surveillance: a questionnaire survey. JMIR Mhealth Uhealth 2016 May 25;4(2):e65 [FREE Full text] [CrossRef] [Medline]
    28. Data.worldbank. 2015. Internet users (per 100 people) in Latin America and the Caribbean   URL: http://data.worldbank.org/indicator/IT.NET.USER.P2?locations=ZJ [accessed 2017-10-31] [WebCite Cache]
    29. Data.worldbank. 2015. Mobile cellular subscriptions (per 100 people) in Latin America and the Caribbean   URL: http://data.worldbank.org/indicator/IT.CEL.SETS.P2?locations=ZJ [accessed 2017-10-31] [WebCite Cache]
    30. Orellano PW, Reynoso JI, Antman J, Argibay O. [Using Google Trends to estimate the incidence of influenza-like illness in Argentina]. Cad Saude Publica 2015 Apr;31(4):691-700 [Article in Spanish] [FREE Full text] [Medline]
    31. Frontlinesms. 2017.   URL: http://www.frontlinesms.com/ [accessed 2017-07-28] [WebCite Cache]
    32. PAHO. Ais.paho. 2010. Influenza & ORV country cumulative analysis   URL: http://ais.paho.org/phip/viz/ed_flu.asp [accessed 2017-04-30] [WebCite Cache]
    33. German RR, Lee LM, Horan JM, Milstein RL, Pertowski CA, Waller MN, Guidelines Working Group Centers for Disease Control and Prevention (CDC). Updated guidelines for evaluating public health surveillance systems: recommendations from the Guidelines Working Group. MMWR Recomm Rep 2001 Jul 27;50(RR-13):1-35; quiz CE1. [Medline]
    34. Lee LM, Teutsch SM, Thacker SB, St. Louis ME. Principles & Practice of Public Health Surveillance. Oxford: Oxford University Press; 2010.
    35. Bengtsson L, Lu X, Thorson A, Garfield R, von Schreeb J. Improved response to disasters and outbreaks by tracking population movements with mobile phone network data: a post-earthquake geospatial study in Haiti. PLoS Med 2011 Aug;8(8):e1001083 [FREE Full text] [CrossRef] [Medline]
    36. Yang C, Yang J, Luo X, Gong P. Use of mobile phones in an emergency reporting system for infectious disease surveillance after the Sichuan earthquake in China. Bull World Health Organ 2009 Aug;87(8):619-623 [FREE Full text] [Medline]
    37. Tambo E, Ugwu EC, Ngogang JY. Need of surveillance response systems to combat Ebola outbreaks and other emerging infectious diseases in African countries. Infect Dis Poverty 2014;3:29 [FREE Full text] [CrossRef] [Medline]
    38. Baltrusaitis K, Santillana M, Crawley AW, Chunara R, Smolinski M, Brownstein JS. Determinants of participants' follow-up and characterization of representativeness in Flu Near You, a participatory disease surveillance system. JMIR Public Health Surveill 2017 Apr 07;3(2):e18 [FREE Full text] [CrossRef] [Medline]
    39. Lau EH, Zhang Q, Kwok KO, Wong IO, Ip DK, Cowling BJ. Using health-seeking pattern to estimate disease burden from sentinel surveillance. In: Online J Public Health Inform. 2016 Presented at: ISDS Annual Conference Proceedings; 2015; Istambul, Turkey. [CrossRef]
    40. Wade AG, Crawford GM, McLennan K. The economic impact of influenza-like illness on families living in the UK demonstrated by the collection of online data. Value Health 2016 Nov;19(7):A392. [CrossRef]
    41. Peasah SK, Azziz-Baumgartner E, Breese J, Meltzer MI, Widdowson MA. Influenza cost and cost-effectiveness studies globally--a review. Vaccine 2013 Nov 04;31(46):5339-5348. [CrossRef] [Medline]
    42. Bilcke J, Coenen S, Beutels P. Influenza-like-illness and clinically diagnosed flu: disease burden, costs and quality of life for patients seeking ambulatory care or no professional care at all. PLoS One 2014;9(7):e102634 [FREE Full text] [CrossRef] [Medline]
    43. Noveck BS. Five hacks for digital democracy. Nature 2017 Apr 19;544(7650):287-289. [CrossRef] [Medline]
    44. Prieto JT, Zuleta C, Rodríguez JT. Modeling and testing maternal and newborn care mHealth interventions: a pilot impact evaluation and follow-up qualitative study in Guatemala. J Am Med Inform Assoc 2017 Mar 01;24(2):352-360 [FREE Full text] [CrossRef] [Medline]
    45. Lister C, West JH, Cannon B, Sax T, Brodegard D. Just a fad? Gamification in health and fitness apps. JMIR Serious Games 2014;2(2):e9 [FREE Full text] [CrossRef] [Medline]
    46. Zichermann G, Cunningham C. Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps. USA: O'Reilly Media; 2011.
    47. López MM, Rodríguez-Modroño P. Presiones de oferta y demanda sobre políticas formales de cuidados en Latinoamérica. Rev CLAD Reforma Democracia 2014;60:103-130 [FREE Full text]
    48. Dalton C, Carlson S, Butler M, Cassano D, Clarke S, Fejsa J, et al. Insights from flutracking: thirteen tips to growing a web-based participatory surveillance system. JMIR Public Health Surveill 2017 Aug 17;3(3):e48 [FREE Full text] [CrossRef] [Medline]

    ‎
    CDC: Centers of Disease Control and Prevention
    ILI: influenza-like illness
    IQR: interquartile range
    mHealth: mobile health
    PAHO: Pan American Health Organization
    RSV: respiratory syncytial virus
    SIM: subscriber identity module
    SMS: short message service
    UVG: Universidad del Valle de Guatemala

    Edited by G Eysenbach; submitted 31.07.17; peer-reviewed by K Baltrusaitis, E Tambo; comments to author 07.09.17; revised version received 10.09.17; accepted 04.10.17; published 14.11.17

    Copyright

    ©José Tomás Prieto, Jorge H Jara, Juan Pablo Alvis, Luis R Furlan, Christian Travis Murray, Judith Garcia, Pierre-Jean Benghozi, Susan Cornelia Kaydos-Daniels. Originally published in JMIR Public Health and Surveillance (http://publichealth.jmir.org), 14.11.2017.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on http://publichealth.jmir.org, as well as this copyright and license information must be included.