This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on http://publichealth.jmir.org, as well as this copyright and license information must be included.
Underreporting of foodborne illness makes foodborne disease burden estimation, timely outbreak detection, and evaluation of policies toward improving food safety challenging.
The objective of this study was to present and evaluate Iwaspoisoned.com, an openly accessible Internet-based crowdsourcing platform that was launched in 2009 for the surveillance of foodborne illness. The goal of this system is to collect data that can be used to augment traditional approaches to foodborne disease surveillance.
Individuals affected by a foodborne illness can use this system to report their symptoms and the suspected location (eg, restaurant, hotel, hospital) of infection. We present descriptive statistics of users and businesses and highlight three instances where reports of foodborne illness were submitted before the outbreaks were officially confirmed by the local departments of health.
More than 49,000 reports of suspected foodborne illness have been submitted on Iwaspoisoned.com since its inception by individuals from 89 countries and every state in the United States. Approximately 95.51% (42,139/44,119) of complaints implicated restaurants as the source of illness. Furthermore, an estimated 67.55% (3118/4616) of users who responded to a demographic survey were between the ages of 18 and 34, and 60.14% (2776/4616) of the respondents were female. The platform is also currently used by health departments in 90% (45/50) of states in the US to supplement existing programs on foodborne illness reporting.
Crowdsourced disease surveillance through systems such as Iwaspoisoned.com uses the influence and familiarity of social media to create an infrastructure for easy reporting and surveillance of suspected foodborne illness events. If combined with traditional surveillance approaches, these systems have the potential to lessen the problem of foodborne illness underreporting and aid in early detection and monitoring of foodborne disease outbreaks.
The World Health Organization estimates that each year roughly 1 in 10 people worldwide experience illness after consuming contaminated food [
To augment traditional approaches (via phone, email, forms on department of health websites, fax, etc) to foodborne illness reporting, illness complaints submitted on social media and business review sites have been used by local departments of health for targeted restaurant inspections and foodborne disease outbreak surveillance [
Alternatively, these data can be collected via crowdsourcing, which refers to the collection of information or completion of tasks by a large public audience [
Crowdsourced surveillance of foodborne illness can be useful in several ways. First, crowdsourced reporting of foodborne illness complaints is useful for general monitoring to enable targeted restaurant inspection [
The objective of this paper was to present an efficient and easy-to-use data submission platform—Iwaspoisoned.com—for crowdsourced surveillance of foodborne illness.
Iwaspoisoned.com (
A report can be submitted on Iwaspoisoned.com using the form displayed on the left in
To ensure data quality, human curators review reports for tone and accuracy of restaurant information. The restaurant details, which consist of the restaurant name and location, are validated using the Google search engine. If the restaurant name or location is missing or incorrect, the report is held for further review. We attempt to contact the user to obtain the missing details if the contact information is provided. In the event that we cannot validate the business location, the report is deemed invalid and is not published. Additionally, each description of a foodborne illness experience is carefully read to detect obscene or abusive language or language that suggests an attack on the business. Two examples of the 1700 submitted reports that have been excluded are presented.
Ate a piece of Trident Layers strawberry gum that my friend bought, felt a small (very tiny) trace of fever (mild headache, sore throat, mild loss of appetite) 1 hour later. Ilovehorseyrides.
Location: Bikini Bottom I went to the Krusty Krab a few days ago. I ordered a Krabby Patty that shut was expensive n*gga. I got home and everything was all gucci. Next morning I saw dookie stains all over.my bed sheets and it smelled like ASS nigguh. Damn that sh*t was off chain. I suffered a Fever and I puked on my wifes lap and she bitch slapped me. F*ck Mr Krabs, Krusty Krab? More Like Krusty Crap. People, buy at the Chum Bucket, I hear their new human hot dog is a.must eat.
Example 1 was excluded because of the inclusion of the nonsensical text—“ilovehorseyrides”—in this report and several other reports from the same IP address. All reports containing the nonsensical text were excluded. In Example 2, the restaurant cited was fictional, so the content was deemed inauthentic.
Furthermore, to eliminate spam, we verify IP addresses through IP lookup tools and identify suspicious patterns using a combination of automated and human review. We also consider a combination of timestamp, contact email, IP address, restaurant details, and description to identify multiple submissions of similar reports by the same user. If the duplicate reports are about the same incident and the details do not change, we assume the multiple submissions are because of user error. In such cases, only one report is published.
A report is only posted to the site and considered in review of larger trends after the authentication process. Food poisoning complaints posted on Iwaspoisoned.com are reviewed and compared with other reports to identify patterns in reported symptoms and geography. As foodborne diseases have varying incubation periods, there’s a possibility of individuals implicating the wrong foodservice business; therefore, surveillance is focused on identifying case clusters within a single report or across multiple reports rather than a single case. Some users submit reports while or right after experiencing illness, making it easier to identify and track disease clusters for timely outbreak control. Whenever a potential emerging or ongoing food safety issue is identified, the appropriate health department is contacted via email or phone and provided with details on the implicated business, symptoms, and contact information, if included in the report. The health department is contacted when two or more reports stating that multiple people experienced illness after eating at the same location are received within a span of 7 days.
To better understand user demographics, we developed a survey that asked users about their gender (male/female) and age (under 18, 18-24, 25-34, 35-44, 45-55, 55+). The optional survey ran from February 6, 2016 to August 12, 2016, and was presented to every person who submitted a complaint. A second survey that focused on doctor visits and disease diagnosis was run from Feb 28, 2016 to July 10, 2016. The survey asked two questions: (1) Did you see a doctor? (Yes/No), and (2) If yes, was there a diagnosis? If there was a diagnosis, was the user diagnosed with a foodborne disease or another disease?
In the last two years, the system has assisted in the detection and monitoring of outbreaks associated with at least three major food chains. Our knowledge on early detection involving communication with a health department is limited to outbreaks that were reported in the news. We present three examples: two local outbreaks and one of national attention because there were multiple outbreaks associated with the same chain across the United States. Note that these examples are meant to demonstrate the potential utility of this system and do not in any way suggest that foodborne outbreaks are limited to these restaurants.
Screenshot of the Iwaspoisoned.com website.
As of October 6, 2016, 49,934 unique reports have been collected. Most reports originated from the United States (44,524/49,934, 89.17%), Canada (1070/49,934, 2.14%), United Kingdom (1050/49,934, 2.10%), and Australia (178/49,934, 0.36%). In the United States, the highest number of reports were received for California, accounting for 16.95% (7547/44,524) of all reports, followed by Texas with 7.10% (3161/44,524), Florida with 5.80% (2581/44,524), and New York with 4.24% (1886/44,524;
Of the 44,119 users with precise information on the suspected business, 95.51% (42,139/44,119) and 2.63% (1159/44,119) attributed their illness to restaurants and grocery stores, respectively. The remainder were distributed across other business categories (
The demographics survey had 4616 respondents. Approximately 67.5% (3118/4616) of respondents were under the age of 34, and 60.1% (2776/4616) of respondents were female (
There were 7141 respondents to the doctor visit survey and 88.4% (6315/7141) did not see a doctor. Of those who saw a doctor, approximately 24.3% (201/826), 22.4% (185/826), and 5% (37/826) received a general diagnosis, no diagnosis, or other diagnosis (ie, not a pathogen associated with foodborne diseases), respectively. Approximately 7% (58/826) were either still awaiting results or did not want to share results. Of the remaining 41.8% (345/826), 32.8% (113/345) received a diagnosis of
(A) heat map illustrating the distribution of foodborne reports by state with the legend from red to green, representing most to least. (B) heat map showing the most populous states in red, and the least populous in light green. (C) heat map showing per capita food poisoning reports per state. This is the population of the state divided by the number of reports received. The highest number of reports received per capita is in bright red at 1 for every 5200 people, the least number of per capita reports is in white at 1 for every 18,300 people.
Distribution of reports across businesses.
Business | n (%) |
Restaurant | 42,139 (95.51) |
Grocery/Supermarket/Warehouse | 1159 (2.63) |
Retail host | 419 (1.0) |
Convenience store/Gas station | 130 (0.3) |
Hotel/Resort/Casino | 67 (0.2) |
Packaged food product | 57 (0.1) |
Entertainment/Theme park/Festival | 49 (0.1) |
Airline | 35 (0.1) |
Airport | 18 (0.0) |
Shopping mall | 11 (0.0) |
University/School | 11 (0.0) |
Train station | 6 (0.0) |
Stadium | 6 (0.0) |
Hospital | 5 (0.0) |
Food truck | 4 (0.0) |
Community service/Church | 2 (0.0) |
Cruise | 1 (0.0) |
Total | 44,119 (100.00) |
Survey of 4616 Iwaspoisoned.com users.
Age group in years | Female, n (%) | Male, n (%) | Total, n (%) |
Under 18 | 281 (6.09) | 158 (3.42) | 439 (9.51) |
18-24 | 772 (16.72) | 519 (11.24) | 1291 (27.97) |
25-34 | 832 (18.02) | 556 (12.05) | 1388 (30.07) |
35-44 | 453 (9.81) | 304 (6.59) | 757 (16.40) |
45-55 | 251 (5.44) | 164 (3.55) | 415 (8.99) |
55+ | 187 (4.05) | 139 (3.01) | 326 (7.06) |
Total | 2776 (60.14) | 1840 (39.86) | 4616 (100.00) |
We present three examples in which Iwaspoisoned.com detected clusters of similar illness linked to a restaurant before an official press release by the local health departments. Note, the case studies are meant to illustrate the potential of using this system to supplement state and local public health foodborne disease surveillance systems and in no way suggest that foodborne outbreaks are limited to these restaurants. See the US CDC and Surveillance Food Safety website [
First, in 2015, Chipotle, a major fast food company was identified as the source of several foodborne disease outbreaks across the United States. Between August and December of 2015, single- and multistate outbreaks were reported in California, Massachusetts, Minnesota, Wisconsin, Delaware, Illinois, Maryland, Kansas, New York, Oklahoma, Ohio, Oregon, and Washington.
On August 21, 2015, three reports citing a total of 7 people experiencing illness after consuming food at a Chipotle restaurant in Simi Valley, California, were submitted on Iwaspoisoned.com. On August 22, 6 new reports were submitted mentioning 23 sick individuals, and on August 23, 37 reports were submitted mentioning multiple sick persons; all implicating the same restaurant. Additional reports were submitted over the next several days, averaging approximately 5 to 10 per day. The persons mentioned in these additional reports experienced illness during the time frame of the outbreak; however, the individuals were either too sick to report in real time or discovered the website a few days post illness. The number of reports implicating Chipotle in Simi Valley increased from 1 to 30 within 2 days. On September 4, approximately 2 weeks after the initial report, the Ventura County Department of Health, California, confirmed a Norovirus outbreak linked to Chipotle.
(A) The monthly percentage of suspected foodborne illness reports implicating Chipotle relative to all reports submitted to the system. Peaks in reporting are noted during the period when multiple outbreaks were reported. (B) Number of suspected foodborne illness reports implicating Chipotle, and Restaurant Chain A and Chain B, which have similar and greater than five times more store locations compared with Chipotle, respectively. The number of foodborne illness reports implicating Chipotle is significantly higher.
Data submitted to Iwaspoisoned.com suggested that as early as 2014, foodborne illness linked to Chipotle represented a significant proportion of reports submitted to the site (see
Second, on March 13, 2016, Iwaspoisoned.com received four reports citing 20 people becoming sick after eating at an Applebee’s in Corunna, Michigan, and alerted the Shiawassee County Health Department. On March 22, approximately a week after the initial report, the health department confirmed a Norovirus outbreak linked to that Applebee’s restaurant.
Third, in late December 2016, Iwaspoisoned.com contacted the Tacoma-Pierce County Health Department about a potential foodborne outbreak after receiving multiple foodborne illness reports associated with a Melting Pot franchise in Tacoma, Washington. Tacoma-Pierce County Health Department conducted an investigation and subsequently shut down the franchise for 24 hours.
The integration of social media and other digital data sources with traditional foodborne illness surveillance systems has the potential to improve foodborne illness surveillance in the United States. Data submitted on Iwaspoisoned.com can alert public health officers to clusters of foodborne illness reports as noted in the three case studies presented. Similar participatory surveillance systems developed for influenza-like illness have been shown to be useful in assessing influenza-like illness trends, risk factors, estimating attack rates, and care-seeking behavior [
Though crowdsourced surveillance has many advantages, there are some limitations to the use of these systems for public health surveillance. First, recruiting and maintaining participants is a major challenge. A study comparing different methods for recruiting participants suggested that offline enrollment is more effective than Internet-based campaigns [
A second limitation is building a nationally representative sample. The demographics of users in the Iwaspoisoned.com system are skewed toward younger age groups; 67.5% are under the age of 34. We do not have data on the proportion of individuals who experience and report foodborne illness in each age category, therefore we cannot quantify representativeness. Younger and older persons who are at risk of more severe reactions to foodborne diseases might be underrepresented. Furthermore, studies suggest that representativeness in Internet-based systems and data sources for disease surveillance are influenced by factors such as gender, education, and income [
A third limitation involves the influence of news on disease reporting on social media and similar platforms. Though social media can aid in the early detection of disease outbreaks, it can also lead to over- and underreporting during outbreaks, which in some cases correlates with trends in news coverage during an outbreak [
A fourth limitation is the validity of reports. Although there are several processes in place to assess the validity of foodborne illness complaints, these are not ensured to be completely effective. Therefore, Iwaspoisoned.com continues to refine the validation process and emphasize a focus on clusters of reports in the same locality or implicating the same business.
Despite these limitations, there are several advantages to using crowdsourcing for foodborne illness monitoring. Data from Iwaspoisoned.com suggests that the main pathogens reported by users with an official diagnosis of a foodborne disease are the same as those reported as major causes of foodborne illness in the United States. By enabling real-time reporting of illness, public health departments can encourage affected individuals to seek medical care and diagnosis.
Additionally, data from other platforms such as Yelp.com and Twitter.com can be integrated with data from Iwaspoisoned.com to track trends in reporting, identify changes in trends that are common across platforms, and correct for limitations in the individual sources. Furthermore, integrating data from these systems with traditional approaches to disease surveillance has the potential for mitigating the underreporting problem and aiding in timely reporting during outbreaks and disease burden estimation.
Anyone can submit a report on Iwaspoisoned.com; however, the platform currently operates only in English. Future iterations would include additional languages. Additionally, the system will be updated to enable real-time geo-positioned location-based alerts for public health officers. Crowdsourced disease surveillance through systems such as Iwaspoisoned.com uses the ease and familiarity of social media to create an infrastructure for easy reporting and surveillance of foodborne illness events. Collaborations with local public health departments and the food service industry can improve data collection through such systems and create opportunities for educating the public regarding the importance of food safety and disease reporting.
Centers for Disease Control and Prevention IP: Internet Protocol
Elaine O Nsoesie is supported by a grant from the National Library of Medicine of the National Institutes of Health (Award Number R01LM011965). We thank Ananya Sureshkumar for her assistance during the review process.
Patrick Quade is the founder of Iwaspoisoned.com