Yahoo is famous for saying that Hadoop, the data-storage-and-processing platform it helped develop into a big data juggernaut, is “behind every click” on the site. But as the world once again starts analyzing Yahoo’s myriad woes after Sunday morning’s ouster of embattled CEO Scott Thompson, I’m left wondering if its investment in Hadoop didn’t aid in the company’s demise — even if it’s a way down the long list of Yahoo’s mistakes.
While Hadoop might have given Yahoo a competitive advantage early on by letting it better analyze user behavior, it also leveled the playing field over time. For years, Yahoo has used Hadoop to better personalize the content its visitors see, both ads and articles, and now so does everybody else. In the meantime, however, Yahoo failed to evolve its business at the same pace as it did its analytic capabilities.
Big data is great, but it’s not a business model in and of itself. Yahoo might as well have just told its competitors, “We still suck, but we spent a lot building a great technology to help you compete.”
The company that has spent the last couple months suing Facebook over patents — even threatening some core components of the Facebook’s infrastructure — used to be on the forefront of open source technologies for webscale operations. Hadoop especially, which Doug Cutting created while a Yahoo employee and then released to the Apache Software Foundation, has always been a point of pride at Yahoo. The company used to maintain its own open source distribution of the platform, and even still ex-Yahoo employees now with Hadoop startup Hortonworks cite the overwhelming majority of the project’s code contributed by Yahoo employees.
But whatever benefits Hadoop brought to Yahoo in terms of letting it better targeting the right content to the right visitors, it also brought them to everyone else. If you talk to a company building an ad-targeting or web analytics technology, chances are Hadoop is somewhere in its technology stack. The aforementioned Hortonworks, which spun out of Yahoo last June, is now peddling Hadoop software and services to the world. A new startup called InsightsOne, also comprised of former Yahoo data experts, is selling a technology for helping marketers reach consumers wherever they happen to be on the web.
The big problem for Yahoo is that, increasingly, users and advertisers want to be everywhere on the web but at Yahoo. Maybe that’s because everyone else that’s benefiting from Hadoop, either directly or indirectly, is able to provide better a better experience for consumers and advertisers alike. Facebook has been particularly adept at using and developing Hadoop to analyze behavior, and — despite potential privacy problems on the horizon — it’s eating Yahoo’s lunch in terms of attracting advertisers and eyeballs.
Not that Yahoo should be blamed for anything it did with regard to Hadoop; rather, it should be commended. It has changed the face of data management on the web like no technology — whether from Yahoo, Google, Facebook or whomever — likely will for some time. But I can’t think of Yahoo and Hadoop without thinking about Sun Microsystems, Java and MySQL, and we all know how that story ended.
Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.
- Dissecting the data: 5 issues for our digital future
- Defining Hadoop: the Players, Technologies and Challenges of 2011
- Infrastructure Q1: Cloud and big data woo enterprises
