The irony is keenly felt by virtually every media entity, whether it is a tiny web-native outlet or a giant traditional publisher: The Internet, the most easily — and frequently — measured medium in the history of humanity, is also one of the most difficult when it comes to getting a straight answer about who’s consuming content and when. That conundrum is at the center of a new report from the recently launched Tow Center for Digital Journalism at Columbia University, entitled “Confusion Online: Faulty Metrics and the Future of Digital Journalism.”
The report notes that even among the leaders in web measurement such as comScore and Nielsen, there can be widespread disagreement about how many visitors a site gets, how much time they spend there, and plenty of other important metrics that might help media websites do their jobs better. In May, for example, comScore said the Washington Post’s website had 17 million unique visitors, while Nielsen said it had fewer than 10 million — a difference of more than 40 percent. Estimates from the two companies of Yahoo’s traffic differed by 34 million, or approximately the population of Canada.
As the report describes, these gaps stem from differences in the way each measurement firm pulls in the data it uses. Some companies have software that users download and install, while others use information that comes directly from Internet providers. Some actually still sample web users by calling them on the telephone and asking them what sites they visit regularly. And all numbers being used that come from the servers themselves are polluted to some extent by the effect of “click-bots” and other scams that can inflate traffic counts, in some cases by huge percentages.
It’s not just comScore and Nielsen; there’s also Omniture, which many larger media companies use, as well as Google Analytics and other services that measure traffic directly from a site’s servers, as well as services like Hitwise and Compete that take data from ISPs and other sources. There are also newer analytics tools that can give publishers an almost real-time look at the activity on their site, such as Chartbeat. While some news sites are excited by the potential of all this data, others quoted in the report say there’s so much conflicting information, it’s actually making their jobs harder rather than easier, particularly when it comes time to talk to advertisers:
Tom Heslin, senior vice president and executive editor of the Providence Journal, calls this the “irony of expectations”: neither publishers nor advertisers have been able to keep up with the flood of data. “Our biggest challenge is to simplify solutions for our clients, even for national advertisers,” he explains. “The development of metrics has far outstripped knowledge of ad buyers and sellers.”
As this comment makes clear, the biggest issue with metrics isn’t that newspapers or websites can’t figure out what their readers want to read — although that is part of the problem. The big issue, in financial terms at least, is that most publishers rely on advertising for the bulk of their revenue, and there’s no simple standard for showing traffic or readership online the way there is in the dead-tree world, where NADBank and other measurements are accepted (although they also have their flaws). That leaves media outlets in a quandary, says the Tow report:
Uncertainty about audience measurement hinders online ad spending, with buyers and sellers of media favoring incompatible metrics [and] increasingly, the decisive information resides not with the publisher but in the databases of intermediaries such as ad networks or profile brokers.
So what is the future of online measurement? The report doesn’t come out and say this, but it probably looks a lot like the present: multiple competing measurement sources, each of them flawed in a slightly different way, so that most publishers have to throw as many different numbers as they can into a hat, then average them all out and hope for the best. The report’s authors say there’s a chance that Nielsen and comScore might merge to become the dominant player and thereby, the de facto measurement standard, or that publishers and advertisers will settle on Google Analytics as the main arbiter of traffic numbers, but both of those seem relatively far-fetched.
In all likelihood, digital media will have to continue muddling along as best it can, for better or worse, tormented by the fact that in a medium where measurement is so easy, measurement by itself can mean very little.
Related GigaOM Pro content (sub req’d):
- Why Google Should Fear the Social Web
- Lessons From Twitter: How to Play Nice With Ecosystem Partners
- What We Can Learn From the Guardian’s Open Platform
Post and thumbnail photos courtesy of Stuff on My Cat
