How social media is making polling obsolete

Companies are high on social media for a number of reasons, but perhaps chief among them should be that social platforms provide the opportunity to create focus groups at a scale never before possible. Millions of people talk about all sorts of things online, and with the right systems and algorithms in place, it’s possible to decipher how they actually feel about the topics they’re discussing. If you want to know how the web-savvy world feels about a product, movie, team, you name it, millions of data sources should trump interviewing a few hundred people in malls across the country.

IBM has been going out of its way to illustrate the insights that can be gleaned from social media by analyzing and scoring fan sentiment on Twitter around the the World Series, Super Bowl and Hollywood awards. The results have been pretty telling, especially if you assume the thoughts of a few million people speaking freely are more telling than those of a substantially smaller number of people willing to pick up the phone or waste 30 minutes of their day answering questions. For customers, IBM sells a version of its Cognos BI product tuned specifically for social media, and it believes social-media analysis will help it do $ 16 billion in analytics revenue by 2015.

But IBM isn’t alone in putting big data techniques to social media streams.

Dachis Group is an Austin, Texas-based company that launched in 2008 with the sole focus of telling its large-business customers everything they want to know about their social media efforts. Initially, CTO Erik Huddleston told me, that meant a lot of analysis to determine whether marketing campaigns were resonating with consumers, but now many companies constantly monitor social media to gain general insights into how consumers feel about their brands overall. And Dachis Group will do just about anything to get companies the information they desire.

Step 1: Take lots of data

At a high-level (although you can drill pretty deep) Dachis Group provides its Social Business Index, which monitors about 30,000 brands and ranks their overall social-media status. The idea is to let companies see what the leading brands in their space are doing so those companies can try to replicate that success. Huddleston said about 40 percent of the Fortune 100, 60 percent of the Fortune 500 and 30 percent of the Global 2000 are registered Social Business Index users.

Although its intuitive enough on the surface, the slick user interface belies some serious analytics underneath. Huddleston said the Social Business Index was built atop a research project that involved analyzing how top brands were engaging on social media and then correlating that engagement to desired business outcomes such as brand awareness, love, loyalty and customer satisfaction. It then went a step further by drilling down to determine what tactics worked best in different geographies and how subsidiaries performed. Further, Dachis Group isolated and analyzed external events to determine how layoffs, mergers, product launches and other things might affect a company’s social buzz.

Huddleston compares analysis this detailed to building a beaver dam: “[Y]ou see where the water’s leaking out and you keep plugging it until you have a good handle on the stream.”

Step 2: Drill down

However, while the Social Business Index is something of a one-size-fits-all SaaS offering, Dachis Group also does intense consulting work for its customers and has discovered a lot of about consumer thinking in the process. For example, Huddleston explained, Dachis Group was monitoring pre-Black Friday activity for a retail client and saw the Target opening-hours trainwreck unfolding before it occurred. Consumers’ perceptions of stores “was exactly correlated to opening times,” he explained.

Ironically, while Target annoyed customers with its Thanksgiving-night store openings, it won them over with its “Christmas champ” marketing campaign. While most retail mentions on Black Friday were about deals, people were talking up a storm about Target’s commercials.

Erik Huddleston

Huddleston calls this sort of work more like Minority Report in that it’s able see certain trends build up steam into national outrage, but said Dachis Group also does a fair amount of CSI-type work. If a client wants to determine why a particular video went viral or why one brand of lipgloss is selling better than others in a certain geography, Dachis will do some investigating. This can be a particularly interesting process, and one that might catch on as more companies try to figure out the whys (subscription required) after their big data efforts have uncovered that whats.

Essentially, Huddleston explained, Dachis Group will isolate a particular subset of social media users and monitor they’re interactions over an extended period. By doing this, it can look for specific mentions of the product, video or whatever in question, but can also use semantics to identify greater cultural influences or other trends that might be driving a particular behavior.

As one might expect, all this analysis requires a serious big data architecture. Huddleston said Dachis Group runs hundreds of virtual servers across the world on AmazonWeb Services, as well as a who’s-who list of next-generation data technologies. That includes Cassandra as a backend NoSQL data store, as well as lots of Hadoop — including the Apache Mahout machine-learning project — and custom algorithms around natural-language processing and sentiment analysis. In order to ensure it got everything right, Huddleston said Dachis Group hired contributors from the various Apache projects it uses.

IBM, of course, has all sorts of its own tools to power analysis, ranging from Hadoop for number-crunching to Watson for natural-language processing to SPSS for predictive analytics.

Step 3: Kill polling?

Which brings us back to the initial question: As social media usage grows and the ability to analyze that activity gets better, are fan voting, focus groups and telephone polling on their way out as the primary methods for finding out what “the people” really think? Dachis Group has actually provided a little food for thought — an infographic comparing its findings to those of USA Today‘s AdMeter for rating the popularity of Super Bowl commercials. You be the judge as to which is more accurate.