Twitter is an endless stream of stories, tidbits and anecdotes. Separately, they are singular narratives — but in big data form, they can actually tell us a lot about the world we live in.
The latest example: Twitter actually has a pretty good idea of which restaurant gave you a stomache ache, and why.
A research report from the University of Rochester (via Vice) proves that a bit of careful curation of a firehose feed of tweets can actually pinpoint when and where someone got food poisoning — and it’s scarily accurate with its results.
The researchers’ system, called nEmesis, “listened” to 3.8 million tweets from more than 94,000 unique users living in New York City. The program specifically targeted tweets with keywords indicating that a user was dining at a restaurant, and isolated GPS data (provided voluntarily by the user) to confirm it. From there, the system tracked the user for 72 hours to see if any keywords pop up to indicate illness.
The program also reverse-engineered food poisoning cases, picking out keywords from food poisoning and then goes backward to find probable locations of dining through GPS.
In total, the program traced 23,000 restaurant visitors and found 480 cases of food poisoning. Even more impressive, when compared the NY Health Department’s grading of those restaurants, researchers found that there was a direct correlation between the restaurant’s food safety standards and the diners’ poor experience.
In short, if the Health Department found it unsafe, the diners were often at risk. According to the researchers, roughly one-third of Twitter data reliably predicted the rating of the restaurant.
Of course, Twitter’s method isn’t perfect — in order to even collect the data, researchers had to babysit the program and subtly tune it to filter instances of food poisoning versus other common illnesses. But it does show the potential for crowdsourcing to accurately monitor public health.
Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.
- Best practices in optimizing content for social engagement
- Examining the rise of crowd labor platforms in 2012
- 12 tech leaders’ resolutions for 2012