Last month, the University of Southern California’s Annenberg Innovation Lab unveiled its Film Forecaster, a tool that tries to predict upcoming blockbusters by the amount of buzz around a movie on Twitter. It’s not the only one of its kind; others like Boxoffice.com have also been tapping Facebook and Twitter data to predict movie success.
But what’s interesting to me about the project isn’t just the sentiment analysis applied to movie tweets, but how easy it was for Annenberg’s Innovation Lab to implement it. The lab, which is sponsored by IBM, used IBM’s new Big Sheets application, which became available publicly last month as part of Big Insights, and has been applied to other projects like tracking the U.S. election and the Egyptian uprising. The film forecaster sounds like a big undertaking for USC, but it really came down to one communications masters student who learned Big Sheets in a day, then pulled in the tweets and analyzed them.
Big Sheets works like a big spread sheet and can be used to gather and analyze petabytes of unstructured data. Because it works with a familiar paradigm, it’s easy for people to use and can be partnered with programs like ManyEyes to visualize information. That’s what happened in the case of the film forecaster, which started in May and looked at tweets over a 24-hour period. The lab was able to gather between 250,000 and 500,000 tweets for each analysis and break them down into positive and negative messages using a lexicon of some 1,700 words. The student in charge was able to create a visualization a day later based on the data.
“I think data analytics is incredibly important going forward,” said Jonathan Taplin, director of the Innovation Lab. “Tools like BigSheets allow us to do something very important — sentiment analysis — without using a lot of data-centric people involved in the project. It’s incredibly easy to use and very efficient too.”
What this shows is that with the rise of big data, we’re also seeing the emergence of really powerful but simple tools that can democratize data analytics and business intelligence. Big data won’t necessarily be handled by just data scientists; it can be wielded by non-technical people. That’s a powerful idea, because it suggests a world in which we can all be data jockeys.
Rod Smith, VP of emerging technology for IBM, said companies are increasingly looking to mine the unstructured data from Twitter and Facebook, where their consumers are. He said with tools like BigSheets, it’s becoming easier for domain experts and business-side people to delve into data analytics, without the help of IT teams and data gurus. Analytics, he said, will become a core competency for workers as the tools become easier to handle.
“It will become almost second nature; you’ll have ability to get information and do this type of sentiment analysis,” Smith said. “Someone who is skilled in other areas and doesn’t know the constructs of data will just know where to get it and get insights from it.”
IBM won’t be the only one offering simple tools for handling and visualizing data. Stacey recently wrote about companies like Tableau, Karmasphere and Microsoft, which are also building simpler tools for data analysis.
Decision-makers and everyday workers will also be on the frontlines of gathering and analyzing the data. And as more of this analysis goes real-time, any lags in drawing insights collapse. The promise of big data and these easy-to-use tools is in empowering more people to make fast decisions drawn from real data.
Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.
- Infrastructure Q1: IaaS Comes Down to Earth; Big Data Takes Flight
- Defining Hadoop: the Players, Technologies and Challenges of 2011
- Putting Big Data to Work: Opportunities for Enterprises
