Why Facebook might put Blu-ray to use on big data

Sometimes, when Facebook taketh away, it also giveth. While the Facebook-founded Open Compute Project is working hard to drive margins and any sense of stability entirely out of the server business, Facebook itself is telling the storage industry where it should be focusing its attention next — on a wide array of storage options to address a wide array of big data applications. Essentially, Facebook is Goldilocks to the storage industry’s three bears — it wants its storage just right.

Jay Parikh, Facebook, Structure 2012

Stacey Higginbotham (left) and Jay Parikh (right) JULIADEBOER PHOTOGRAPHY http://www.juliadeboer.com

During a keynote at the Open Compute Summit on Wednesday, Facebook VP of Engineering Jay Parikh laid out his vision for a storage architecture that encompasses everything from hard disk to flash to — yes — Blu-ray in order to meet the needs of the company’s growing number of applications. The idea (and one that we’ll be discussing at Structure: Data in March) is that storage companies need to think more broadly about how companies might use their technologies in order to serve applications that have different requirements about how they access and deliver data.

Facebook’s concerns are especially relevant in the era of big data, when companies are amassing always-growing volumes of data that they might not want to, or be able to, throw away. So, while database performance might be critical for applications such as wall posts on Facebook — thus necessitating all-flash servers like Facebook’s new Dragonstone design — it’s markedly less important when it comes to storing petabytes worth of user photos that haven’t been accessed in two years. In the middle might fall something like Hadoop, which Facebook uses as a massive data warehouse that, really, just needs to be fast enough to be useful.

Parikh, for what it’s worth, likes to use a car analogy instead of a fairy tale one. Discussing the company’s desire to use more flash at our Structure Europe conference in October, he said the current process of flash deployment is “like adding a Ferrari engine to your server.” During his presentation at the Open Compute Summit, he compared hard disks to minivans, flash to sports cars and pined for something like a Toyota Prius to be the third car in the garage.

Facebook's new Dragonstone server.

Facebook’s new Dragonstone server.

Although Blu-Ray might be a stretch at this point, he told me during a hallway chat after the session, it actually holds some promise. After all, it’s fairly fast, it’s very low power and it’s highly durable. If he were producing Blu-ray disks or flash drives, Parikh said, he’d try to consider a world where hard drives don’t exist and then start thinking about how to expand the storage medium’s utility to fill that void in a way hard drives never could.

And where Facebook goes on infrastructure, large industries and their IT budgets often follow. Facebook’s Open Compute server designs might have seemed somewhat crazy two years ago, but now large banks and cloud providers are on board evaluating and even building their own servers. Server makers such as Dell saw the writing on the wall and are getting behind Open Compute early in order to meet demand for this type of hardware in the future.

As more companies look to match their storage media with the specific needs of myriad types of data applications, the storage vendors and manufacturers that provide this capability might be the big winners. Companies might like to see hard drives go the way of tape drives, but that can’t happen until something better — and better suited to a big data world — take their place.

Feature image courtesy of Shutterstock user Nomad_Soul.


GigaOM