Is Carrier IQ a big data mercenary?

A video posted by an Android developer has turned into a scandal that could envelope the whole wireless industry. Since developer Trevor Eckhart first revealed the details of how a mysterious keystroke-logging application created by Carrier IQ tracked every action performed on Android phone, operators, handset vendors and even the almighty Apple have been implicated in the plot. But it’s not just the obvious wireless players that see value in Carrier IQ’s covertly collected data.

Media-measurement company Nielsen is tapping into that information pipeline as well, which raises the questions of whom else Carrier IQ is selling its information, too. Providing a carrier with anonymous performance metrics is one thing, but selling data compiled from to a third-party with no relationship to the customer or the network Carrier IQ is monitoring. Carrier IQ claims to be offering a service to the operators to help them optimize their networks, but it may well be a big data mercenary selling information on all kinds of mobile consumer behavior to the highest bidder. It might even be playing both sides.

The story so far

AT&T and T-Mobile USA are just latest to admit it received data from Carrier IQ on the behavior of its customers’ smartphones, though, like Sprint, they claimed it used that information solely for network optimization purposes. Verizon is the only major U.S. operator untarnished. Apple copped to installing Carrier IQ’s software into all of its iPhones before the release of iOS 5. HTC and Samsung acknowledged implementing Carrier IQ, but only at the behest of their carrier customers, which didn’t prevent them from getting slapped with class action lawsuits.

Lawyers have compared Carrier IQ’s covert digital snooping to illegal wiretapping. Carrier IQ has even attracted the attention of Congress. U.S. Senator Al Franken (D-Minn.), who chairs the Senate subcommittee on Privacy, Technology and the Law, sent a letter Thursday to Carrier IQ President and CEO Larry Lenhart asking some poignant questions about how and for whom Carrier IQ collects its data.

Taking a cue from Franken, let’s ask some of those same questions. Moving beyond what data Carrier IQ is collecting, which has been covered extensively by Eckhart and subsequent stories, lets’ explore why Carrier IQ is collecting information from millions of smartphones and more importantly who its selling that data to.

A bizarre big-data triangle

Based on the patterns of admissions and denials we’re seeing around the industry, as well as some background conversations with some industry sources, it looks like Carrier IQ is two-headed beast: one head being its covert handset software and the other being its measurement and analytics service. Certain handset makers like HTC and Samsung are installing the app on many, if not all, of their smartphones at the root layer, but those handsets aren’t necessarily the customers for the analytics service. In fact, both HTC and Samsung deny that it receives any of the data collected.

Then who does? AT&T, T-Mobile and Sprint are three, as was Apple, but another is Nielsen. In October, Nielsen signed on as a Carrier IQ partner saying it would use the company’s technology to help “measure the performance of mobile services, networks, and devices” and “gather actionable intelligence on the performance of mobile devices and networks.”

That sounds very much like what both AT&T and Sprint are saying. We reached out to Nielsen to ask what exactly they’re doing with Carrier IQ data. Here’s the email response form VP of Global Communications Marivi Lerdo de Tejada:

“Nielsen and Carrier IQ announced an alliance in October 2011 to explore potential ways to measure mobile services, networks and devices, exclusively using opt-in panels and in accordance with Nielsen’s stringent privacy standards. To date, we continue to explore these opportunities, with neither any work for clients initiated, nor any panels created.”

If Neilsen sticks with its opt-in policies, it won’t get into the trouble the carriers and handset vendors appear to be in, but are there other market research companies that might not be so scrupulous? Carrier IQ has no qualms with selling carriers their own customers’ data without those customers’ permission. Could it sell the data it collects from AT&T, Sprint and T-Mobiles’ customer to another market analytics firms. Could it sell AT&T’s data to Sprint and vice versa?

Late Thursday, Carrier IQ broke its silence, putting out a statement explaining what data it collects and what it does with it. Here are some excerpts:

“While a few individuals have identified that there is a great deal of information available to the Carrier IQ software inside the handset, our software does not record, store or transmit the contents of SMS messages, email, photographs, audio or video. For example, we understand whether an SMS was sent accurately, but do not record or transmit the content of the SMS. We know which applications are draining your battery, but do not capture the screen.

“… Carrier IQ acts as an agent for the Operators. Each implementation is different and the diagnostic information actually gathered is determined by our customers – the mobile Operators. Carrier IQ does not gather any other data from devices.”

Carrier IQ claims to counting the frequency of actions, not the content of the actions itself and that whatever data it sends back to is servers is encrypted and personal information protected. But Carrier IQ isn’t being quite so honest about who is customers or potential customers are. Right there on its home page, Carrier IQ says it gives handset manufacturers as well as wireless operators “unprecedented insight into their customers’ mobile experience.” Nielsen certainly isn’t a wireless operator.

This isn’t just about a few carriers keeping tabs on their customers. This is big data for the mobile world – massive databases of consumer behavior delving into when, how and in what manner we use our devices. By Carrier IQ’s own admission, its software is embedded in more than 150 million handsets. There are plenty of companies that would find that information enormously useful. The problem is Carrier IQ never got permission from all of these smartphone users to collect that data, never told them it was gathering it, and never provided a way of opting out.

Who gave Carrier IQ permission?

Carrier IQ couldn’t just do this on its own, covertly installing rootkit software into millions of phones without anyone’s knowledge. It had to have the cooperation of operators like Sprint and AT&T and of the handset manufacturers that built their devices. HTC and Samsung, are pointing fingers directly at the operators. Take the statement circulating from HTC, which we first saw on Bright Side of the News:

“Carrier IQ is required on devices by a number of U.S carriers so if consumers or media have any questions about the practices relating to, or data collected by, Carrier IQ we’d advise them to contact their carrier. It is important to note that HTC is not a customer or partner of Carrier IQ and does not receive data from the application, the company, or carriers that partner with Carrier IQ. HTC is investigating the option to allow consumers to opt-out of data collection by the Carrier IQ application.”

But if the operators are entirely to blame, how is that Nokia, Google and RIM can claim they don’t use Carrier IQ’s software? It may have been harder to drill down into Nokia and RIMs’ operating system, sure, but Carrier IQ was able to work with Apple to embed its software deep into the iPhone. Plus, if RIM and Nokia can turn down Sprint and AT&T, why can’t HTC and Samsung? It’s not as if Nokia market share is so strong in the U.S. it casually deny a software customization request from AT&T, one of the world’s largest GSM operators. And though RIM claims to be Carrier IQ-free, that hasn’t stopped AT&T, T-Mobile and Sprint from selling plenty of BlackBerrys.

I’m not fully convinced that Carrier IQ’s intent is as evil as the deluge of recent coverage makes it out to be, though its methods are both suspect and scary. But if it’s a conspiracy you’re looking, then there are plenty of possible conspirators. Carrier IQ didn’t do this alone. To drill this far down into the habits of mobile consumers, companies up and down the wireless value needed to be complicit.

Image courtesy of flickr user alancleaver

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.