By Dede Vilven, Enteric Epidemiologist, Salt Lake County Health Department (SLCoHD)
The Salt Lake County Health Department (SLCoHD) serves approximately 1.1 million people in the greater Salt Lake City area. To keep the population safe and healthy, SLCoHD epidemiology staff review and analyze several data streams to identify the cause of foodborne illness outbreaks, as seen in the figure below.
Since 2015, SLCoHD has also used the crowd-sourced review forum Yelp to identify reports of foodborne illness. SLCoHD created an innovative tool allowing the enteric epidemiology group to add Yelp illness reports into their existing data surveillance system. This streamlined process helped the group identify enough new complaints to justify the staff time and resources invested.
SLCoHD was not the first to use social media to identify foodborne illness reports. The New York City Department of Health and Mental Hygiene (DOHMH) collaborated with Columbia University and Yelp on a pilot project in 2012-2013. The project was deemed successful because about half of the reviews identified by their algorithm described an event consistent with foodborne illness. The May 2014 Morbidity and Mortality Weekly Report (MMWR) report about the project is available here.
In another pilot project, the Chicago Department of Public Health used Twitter to search for messages about food poisoning. Within 10 months, project staff members responded to 270 tweets and provided links to the FoodBorne Chicago complaint website. A total of 193 complaints of possible foodborne illness were submitted through FoodBorne Chicago, and 133 restaurants in the city were inspected. The August 2014 MMWR about the project is available here.
After seeing the success of these other projects, SLCoHD held meetings in January 2015 with stakeholders from epidemiology, food protection and administration to discuss the feasibility and objectives of the implementing a Yelp surveillance project. SLCoHD considered partnering with the University of Utah to take a similar approach to the New York City DOHMH, but ultimately the health department developed its own innovative methodology. Using Python software and Yelp’s application programming interface (API), a skilled epidemiology staff member developed a program that allowed foodborne illness-related reviews to be identified without external agreements or financial commitments.
A six-month pilot period returned good initial results, and the project was given a green light with stakeholders. During the combined nine months of the pilot and implementation, 57 Yelp reviews consistent with foodborne illness were identified, resulting in four requests for inspection of implicated food establishments sent to the Bureau of Food Protection.
During this same nine-month period, SLCoHD received the following foodborne illness complaints from other data streams:
- 199 from Utah Department of Health’s “Igotsick” website
- 149 from the SLCoHD Bureau of Food Protection
- 15 from the Utah Poison Control Center
- 15 from the SLCoHD Bureau of Epidemiology
This total of 378 non-Yelp complaints led to 37 foodborne illness-related requests for inspection. The inclusion of Yelp reviews yielded a 15% increase in the number of complaints identified and an 11% increase in the number of foodborne illness-related requests for inspection. Three of the facilities identified through Yelp reviews received onsite environmental health inspections, uncovering 17 critical violations, including improper cooling methods, dirty equipment, and issues with food temperature, cross-contamination, and hand sinks.
This is an example of Yelp review about foodborne illness.
SLCoHD experienced two challenges with using Yelp complaints as illness reports. First, it was not always clear which day the reviewer ate at the restaurant in question; clarity on the date food is consumed is pivotal to enteric epidemiologic investigations. Second, limited follow-up can be done with the reviewer to gain clarity.
SLCoHD continues to use Yelp complaints to add to the food establishment surveillance system and plans to further evaluate the success of the program.
Looking to the future, SLCoHD is exploring the following questions on the role social media in food safety:
- How can we generate citizen engagement? When there is an online report made through social media, how can we best respond?
- How can we create trust with the community and let them know we are not “big brother” spying on them but rather a resource to use when possible foodborne illness has occurred?
- Should we be using social media for surveillance of other public health issues/illnesses?
- Will Yelp continue to be a relevant social media forum for reporting?
