The biggest obstacle to embracing big data? You.

The technological hurdles to embracing big data are well known at this point — McKinsey & Co. predicted a shortage of nearly 200,000 skilled data analysts by 2018 — but an even bigger obstacle to achieving analytics nirvana might be overcoming a strong anti-data corporate culture. It’s easy to grasp the value of adopting big data techniques, but the changes required to listen to the data and, possibly, to transform a company’s business model take a little more work.

Like killing cockroaches when the light turns on

Companies experience two big problems when they make the move to big data, ClickFox Founder and CEO Marco Pacelli told me recently. The biggest problem might be learning to trust the data and act on it rather than operating based on a gut feeling from upper management. When the data starts exposing all of a company’s operational flaws, good executives need to acknowledge them and then figure out a strategy to resolve them.

It’s like figuring out which cockroaches to kill when the light comes on. Pacelli analogized — there might be a lot, but you can only get to a few. His advice: “Don’t do analytics if you’re an executive that can’t make decisions and act on data.” While someone’s gut instinct might be to just tear it down and start over, that’s not always possible for large businesses with legacy operations in place.

The other big challenge is using big data to transform a company’s treatment of customers into company-wide concerns rather than handling each aspect of the customer experience in separate business units or divisions. In many companies, he said, every business operates in its own silo and customer touches within those divisions rarely make it outside those self-imposed walls. However, Pacelli said, someone at the top needs to take and impose a cross-company view of the customer experience, because everything is connected.

The beauty of analytics in the world of big data is that companies can store and analyze all their data — structured, unstructured and semi-structured, company-wide — resulting, in theory, in more-accurate findings and far more flexibility than is possible with traditional analytics techniques such as querying a highly structured and selective data warehouse.

ClickFox is an analytics service provider that processes billions of customer interactions a month for dozens of large clients (including many telecommunications providers), determines the connections between those interactions and spits back recommendations on how to fix the problems it uncovers. (For example, putting information on a web site might make customers happier than making them go through 20 customer service agents to get the same information).

Case study: Sprint

Sprint, it turns out, is something of a poster child for the challenges of the big data transition. Pacelli explained to me how Sprint used ClickFox to help drastically improve customer satisfaction. One of Sprint’s biggest issues, Pacelli said, was complexity: The company used to offer consumers more than 300,000 service options, which not only overwhelmed customers, but also made it very difficult to determine why customers were unhappy.

Sprint has simplified its model and customer satisfaction has skyrocketed, but it was a long road. It took ClickFox’s system about five years to finally uncover all the choices Sprint customers were being asked to make, Pacelli said, and then it was up to Sprint to figure out how to act on what it found. That’s not always an easy thing to do, especially when, as Pacelli noted, his system is recommending about 50 methods a month for Sprint to cut costs or optimize its business, but the company is only able to act on five of those.

And then there’s Sprint’s separate effort to grow an analytics business to complement its core telco business. Speaking on a panel last week at IBM’s Smarter Analytics Leadership Summit, Sprint Innovation and Advanced Labs Director Von McConnell discussed how the company is trying to productize its customer data into advanced analytics products that will show how consumers interact with their telco services.

Business issues aside, one big concern is in the big data infrastructure itself. Analytic systems will start popping up everywhere once the analytics business moves from lab to production, McConnell said, and every division will probably have its own suite of tools. Assuming centralization is impossible, the challenge will be figuring out how to federate those petabytes of disparate data sources so Sprint can actually conduct meaningful analysis on them.

Sprint is a telco, he said, not (yet) an analytics company, so it’s a tough transition. Illustrating how badly things can go wrong with so many systems in place, McConnell pointed to an existing piece of enterprise software within Sprint that has 32 installations across the company, all of which are wholly disparate.

A job done right

When it all comes together, though, a company’s big data effort can be truly transformative. It can, as Pacelli says, take users “down a path of really starting to think differently.” A path where every step along way is informed by the previous steps, as well as where the data suggests about the path’s future route.

Ryan Leslie, Seton Healthcare’s VP of Analytics and Health Economics, who shared the stage with McConnell at the IBM event, offered an example of how data led his hospital system to think differently. Following a CEO mandate to find better ways to detect congestive heart failure early in order to save the exorbitant costs of treatment as the disease progresses, Leslie’s team analyzed a stockpile of data ranging from billing records to patient charts. It found that a distended jugular vein — something that can be spotted during any routine physical exam — is a particularly high risk factor.

In this case, focusing on simple medicine rather than always relying on expensive technology is going to save Seton and hospitals lots of money (not to mention lives). But the finding wouldn’t have been possible using traditional analytic methods or carried out within a single division: While Leslie was focusing on socioeconomic factors, it was Seton’s CEO, who has a medical background, who spotted the correlation.

Feature image courtesy of Heiko Herrmann; phone-line image courtesy of Jonathan Billinger.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.