Gracenote co-founder on ‘iPod day’ and better music through data

It was April 2000 when the team at Gracenote got a call from Apple that would change its business forever. Apple wouldn’t give Gracenote any specifics, but it did offer up some prescient advice: “You need to buy more servers.”

A few years into Steve Jobs’s second stint as Apple’s CEO, the company hadn’t yet reinvented itself as one of the world’s most-important technology companies, but it was a big-enough distribution channel for the two-year-old Gracenote. At that point, Gracenote had built a respectable business collecting and providing metadata for the compact discs that people were ripping onto their computers, and it relied on software partners to get in front of the music consumers doing the uploading. One of those partners was a popular Mac jukebox application called SoundJam MP.

Ty Roberts Source: Gracenote

So, Gracenote Co-founder and CTO Ty Roberts told me during a recent interview, his company heeded Apple’s warning and bought more servers. At some point around that time (details on the date of the acquisition are sketchy), Apple bought SoundJam MP. Then, at MacWorld in January 2001, Apple released the first version of iTunes (based on the SoundJam technology) and grew Gracenote’s footprint by putting it on more machines. In October 2001, Apple released the iPod and changed Gracenote’s life forever.

The holiday season — particularly Christmas morning — provides a clear example of how stark the change was. “We used to call it iPod day,” Roberts explained, because the company’s servers would go crazy as people opened up their new iPods and immediately began ripping CDs onto their computers. The company’s chief scientist would stay up 20 hours a day for 5 days straight to make sure the database didn’t crash under the load.

From that point on, Roberts explained, a graph showing the rate at which people were uploading music to Gracenote would go from a steady incline into a vertical line. At one point the company was getting metadata from — by Roberts’s estimate — literally every CD being ripped onto personal computers. There was so much database traffic — both writing and reading — because Apple didn’t release the first version of the iTunes Store until April 2003; if users wanted to use their iPods, they had to upload music first.

Scaling like the big boys

Today, of course, Gracenote (which Sony acquired for $ 260 million in 2008) is pretty much ubiquitous, at least when it comes to metadata. It has metadata for about 130 million songs — and growing — from all over the world and provides metadata to everything from iTunes to Path to your car’s entertainment console. Even if they’re not available for sale as MP3, if someone somewhere at some point ripped a CD and entered its information, Gracenote has data on those artists and songs.

Its database now gets 15 billion queries a month, or 500 million a day (“We’re probably bigger than Bing,” Roberts joked), and the company’s infrastructure has scaled a few times to meet this demand. What began as a small web database running on a few servers grew into an Oracle environment that provided better performance. And when Oracle became cost-prohibitive because of Gracenote’s expanding scale, it shifted again into a highly optimized system that spans thousands of cores in four global data centers.

Now, GM and VP of Automatic Content Recognition Michael Jeffrey noted, almost everything from the chip level up is optimized specifically for Gracenote.

There’s no “world music” when you’re in the “world”

And this setup lets Gracenote do a lot more than just recognize music listeners’ files and give them the album art. For one, Roberts explained, it lets Gracenote be a global company. “We want to have all the music in the world,” Roberts said, “… because our customers ship their products globally.” In fact, part of the reason it’s now part of Sony is that Sony was distributing Gracenote so widely as part of the music player in its Vaio line of laptops.

In order to ensure that everyone has a natural experience wherever they’re accessing Gracenote, part of the job of the company’s 100-person editorial team is to categorize music hierarchically by locality. So, when a user in Japan uploads a CD and Gracenote returns the metadata, it’s categorized as “rock and roll,” for example, rather than a catch-all category like “world music” that a U.S. user might see.

“We want music to feel like a person in your country actually organized it,” Roberts said, “not some dude from California.”

Better music and television through data science

All that data also makes Gracenote a natural fit for recommending new music, although right now the company prefers to let partners handle the algorithms because recommendations tend to be highly product-specific. For example, the iTunes Genius feature is a pretty run-of-the-mill recommendation engine, but, Roberts explained, Apple places a premium on accuracy because its recommendations cost users 99 cents (or more) a shot. With a subscription service like Spotify, though, trying new music is risk-free, so it can play a little faster and looser with its algorithms.

Because Gracenote is present in so many cars — about 35 million — the company has put a lot thought into how to optimally deliver services there, too. Until drivers can bring their interest graphs and music libraries with them to their cars, he explained, any sort of in-car recommendation engine has to be pretty simple and non-distracting — perhaps like thumbs-up or thumbs-down button on the display that will eventually be able to recognize someone’s tastes.

The company has even developed what Roberts calls “machine listening,” which is the ability of an algorithm to recognize the mood, tempo and other audio attributes of music. This is comparable to what Pandora offers, but Gracenote has data on pretty much any song someone could possibly have, which means it can make even your personal music library that much smarter. One idea the company is tinkering is something Roberts describes as “audio coffee.” Depending on any variety of factors — time of day, location, driving conditions or behavior — the stereo system could pick music that either picks up a driver’s pulse or maybe relaxes him.

For Gracenote’s next chapter, the company is banking tablets to deliver a kick like the iPod did last decade. Gracenote is already working with television partners on real-time ad-swapping and intelligent content recommendations, and now it wants to dive deep into the second-screen world. Its new product called Entourage uses a tablet’s internal sensors to hear the television show or music playing in a room and then surface related content, perhaps from the web — like what Entourage user Zeebox provides — or perhaps produced, interactive material like the SyFy channel delivers via its Sync app.

Two ads for two different viewers.

Later this year, GM and VP Jeffrey said, Gracenote will be doing pilots with some large sports broadcasters around a “cheer and jeer” feature that measures how hard people in a room are cheering for or booing their favorite sports teams. If you’re elated, you might see an ad for season tickets. If you’re sad, maybe it’s an an for beer.

Even Roberts is impressed, especially considering that the company’s first use of audio recognition was to make sure users got the right data for their exact version of a song: “I never thought the recognition would break open these kind of new fields.”