Davos does data

Big data has gotten very big if the talking heads at the World Economic Conference in Davos are talking about it. And they are talking about it.

The big data phenomenon refers to the explosion of data of all types — location coordinates churned out by cell phones and GPSes; machine data from manufacturing gear; consumer data from Twitter and Facebook. Just a fraction of that information resides in traditional databases. It’s not only too much to get one’s head around, it overwhelms traditional database and analytics tools, giving rise to a new generation of cloud computing technologies: the Hadoop data framework, NoSQL databases, and big data analytics.

The overriding issue is finding the right data, applying the right analytics to it, and letting it deal with real-world problems. And, the World Economic Forum’s elite attendees including prime ministers, corporate titans and star academics — purport to address the biggest of big problems: climate change, the income gap, food inequality, public health crises, Iran.

One session, entitled Decoding the Data Deluge, chaired by Jan Hesthaven, a professor of applied mathematics from Brown University, deals with the upside opportunity these huge data sets can enable.

From the course description:

It’s a funny thing about data: If you look up “data” in the literature, it invariably comes laden with negative baggage – words such as “overload” and “deluge”. With terms such as these, no wonder scientists and scholars alike shy away! But instead of resisting large data sets, we should be embracing them. Multinational companies are learning how to work with large data sets, and they’re getting better at it all the time. But mid-size and smaller businesses don’t have that expertise. Yet the largest companies will benefit greatly if smaller businesses, many of them key to their supply, production and workforce, can, too, handle massive data. Such ease of data handling can grease the wheels of global commerce.

That is just one of several sessions touching on the topic. Another, Personal Data: The “New Oil” of the 21st Century, dealt with the proliferation of personal data generated by new intelligent devices, networks and software and how to build a personal data ecosystem that will “spur economic and societal value without undermining privacy and civil liberties.” Another session, Convergence on the Go, attacked the consolidation of data voice, video and cellular communications services and its impact on society.

There is also a report: Big Data, Big Impact New possibilities for International Development which outlines the impact the collection and proper application of big data can have on financial services, education, agriculture, and healthcare.

According to the report:

Data collected through mobile devices, whether captured by health workers, submitted by individuals, or analysed in the form of data exhaust, can be a crucial tool in understanding population health trends or stopping outbreaks . When collected in the context of individual electronic health records, this data not only improves continuity of care for the individual, but it can be used to create massive datasets with which treatments and outcomes can be compared in an efficient and cost effective manner.

Big data is good, except when it’s bad

All of this discussion in the Alps is happening at a time when the opportunity big data opens up faces privacy concerns that are playing out internationally, as discussed by GigaOM’s Derrick Harris here.

The issue of who owns what data is huge and there are factions fomenting to wall-off data because of political concerns. Telcom powers in France and Germany, for example, want to build home-grown data centers that would be impervious to U.S. law enforcement requests for customer data. Their concern is that the U.S. Patriot Act can force U.S. companies running cloud-computing sites in Europe (or elsewhere) to turn over customer data if U.S. authorities suspect terrorist activity. That means American IT powers like Microsoft, Amazon, Google that run cloud operations out of European data centers could be compelled to disclose this information. That specter led some European entities to eschew cloud computing services from U.S. companies. In December, British defense contractor BAE, backed out of a Microsoft Office 365 deal, citing this issue.

On January 18, a U.S deputy attorney general Bruce Swartz tried to calm the waters, saying that the Patriot Act does not supercede existing international trade agreements. A few days later, the Electronic Freedom Foundation, an advocacy group, debunked Swartz’s claim.

While the DOJ may spin its position one way to try to appease foreign audiences, its actual position is quite clear where it really matters: in US courts when it is trying to access subscriber information held by US-based cloud computing services. Indeed, the DOJ’s position in its court filings is that very little, if any, privacy protection is available against US government access to the records of users of US-based cloud computing services.

Whatever happens in this data privacy/protectionism battle, it’s clear that big data is very big indeed. Maybe too big. Big data has been featured in recent issues of mainstream publications including Esquire. A contrarian might say this mainstream hype could also mean the big data bubble is about to burst.

Photo and artwork courtesy of Flickr user World Economic Forum

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.