For Google, keeping search relevant means baking big data into everything

It’s a fashionable practice in the Valley to write off Google’s search business, but the company is putting its big data chops to the test to prove doubters wrong. In a Wednesday morning blog post, Google SVP of Search Amit Singhal announced that Google’s Knowledge Graph is now live across every English-speaking country in the world, and that voice search on mobile phones has been improved to understand user intent. Useful, yes, but the real story is the technology that makes these features work.

For Google, it’s all about collecting and analyzing billions of data points to learn what each one really means. With Knowledge Graph, for example, Google uses a “database of more than 500 million real-world people, places and things with 3.5 billion attributes and connections among them.” It’s those connections that are the key, as they’re what make the system smart enough to know what you’re looking for that wouldn’t naturally show up in a standard keyword search.

Although Google hasn’t come out and said so, I’d imagine the Knowledge Graph utilizes Google’s Pregel graph processing engine. Graph processing and databases are catching on in social networks and other large-scale environments because they organize pieces of data by how they’re connected to one another. Those connections are called edges, and they’d keep Knowledge Graph results both informative and focused because the system knows how closely they’re related in any given circumstance.

This example of a personalized interest graph from Gravity Labs illustrates how one might visualize a graph, in this case the connections between a reader’s perceived interests:

Of course, Google has another tool at its disposal, which is the collective wisdom it’s able to glean from billions of searches every day. So, as Singhal wrote when first explaining Knowledge Graph in May, “[W]e can now sometimes help answer your next question before you’ve asked it, because the facts we show are informed by what other people have searched for. For example, the information we show for Tom Cruise answers 37 percent of next queries that people ask about him.”

Google’s other big announcement today is improved voice search on mobile phones, both Android and iOS. Here’s how Singhal describes the new capability:

You just need to tap the microphone icon and ask your question, the same way you’d ask a friend. For example, ask “What movies are playing this weekend?” and you’ll see your words streamed back to you quickly as you speak. Then Google will show you a list of the latest movies in theaters near you, with schedules and even trailers. … When Google can supply a direct answer to your question, you’ll get a spoken response too.

On Monday, a Google Research blog post noted how the company’s work on neural networks — which it famously used to train a system capable of detecting cats and human faces in video streams — is being used to power speech recognition in the Jelly Bean release of Android. Seventeen-year-old Brittany Wenger recently won the Google Science Fair by building an application atop Google App Engine that uses a neural network to help detect breast cancer.

As one might imagine, however, the big challenge for Google, Microsoft , Apple and everyone else trying to provide intelligent but intuitive user experiences is figuring out how to shape high computer science into easily digestible formats on ever-smaller devices. Search would certainly be a more effective tool if everyone could write complex queries directly against a company’s database, but the trick is making products good enough that we don’t have to. It’s boiling years of machine learning, natural-language processing and neural network research into “you ask a question and your phone spits back the right answer.”

Feature image courtesy of Shutterstock user Sebastian Kaulitzki.



GigaOM