Facebook has flash memory on the brain. The social network, which has helped rethink server design for its operations and is designing a new type of infrastructure from the ground up for storing infrequently accessed photos, is thinking about “more use cases for flash,” said Jay Parikh, vice president of engineering and infrastructure.
In a conversation after our discussion onstage at Structure:Europe on Wednesday, Parikh said that instead of just using flash in places where you need performance, “like adding a Ferrari engine to your server,” the social network was thinking about other places it might be useful. He declined to go into more details but implied there were places one could replace hard drives with flash. Flash is common in data centers where performance matters, such as for database storage and for speeding up access to images. Facebook uses flash from Fusion-io is some areas of its operations where performance is essential.
But flash is expensive when compared to the traditional hard drives that most companies use for storage. Part of that expense comes from the work that goes into making flash memory — which was designed for consumers — dependable enough to handle enterprise data. Flash can be unreliable, and as its storage capacity grows, the number of times it saves or accesses data shrinks. And while the cost of flash is dropping, hard drives are still cheaper on a per byte basis.
But flash memory is also more energy-efficient when compared to hard drives. And several sources have told me that Facebook is building software that will allow it to take advantage of flash that has been rejected for high-performance use cases like memcached servers and instead use it for storing data that might otherwise go on hard drives. The software would map the unreliable areas of the flash memory and then avoid putting data in those spots. Other sources have mentioned that Facebook is building “flash appliances” which could be arrays of the rejected flash drives and the software to manage them.
And I think that kind of appliance or custom-flash array would be perfect for Facebook’s cold storage facility it has been talking up recently. The idea behind the cold-storage facility is that Facebook wants a place to efficiently and cheaply store those 220 billion photos it has — it gets 7 petabytes of new photos each month, Parikh said during our on-stage conversation at Structure.
Since after a few days the desire to access those photos diminishes, Facebook needs a place to put them that’s cheap but still accessible for those times users are feeling nostalgic. Cheap flash-based arrays would be perfect. Because the data in cold storage is infrequently accessed, the diminished read-write capability of the drives is less of a problem. Because flash has no moving hardware it’s not only greener, but its something that can be turned on instantly. In an interview with Wired Tom Furlong of Facebook said the equipment in the cold storage array would be designed to turn on and off when not in use.
So, Facebook officially says it wants to use flash in more places, while sources have been sharing details of Facebook’s flash appliances. I think that adds up to flash being a key component of Facebook’s cold-storage boxes. I reached out to Facebook for this story, but have not heard back.
