How search can solve big data problems

There are many solutions for figuring out how to parse large amounts of data, but LucidWorks CTO Grant Ingersoll has a suggestion: use search. At GigaOM’s Structure:Data conference in New York City Thursday, Ingersoll laid out his case for why search is a big part of dealing with databases and indexes.

“Search should be a critical part of your architecture,” he told attendees. It is a system building block for any large problem you’re trying to solve that requires a ranked set of results. And it doesn’t have to be just text search, it can be for any type of search, he said.

Thinking beyond traditional search features, like keyword search, will help businesses solve those problems more easily too. And it lets organizations bring in many differing kinds of data sources and more effectively combine them.

And organizations that keep records of how people are using search to access their data, they have the best current view into the quality of their data.

His main message was to give search another look. “There have been interesting changes in the way we can model relevance in the search engine,” he said. “If you’ve dealt with last-generation search technology, you should revisit this because there’s a lot more capabilities here.”

Check out the rest of our Structure:Data 2013 live coverage here, and a video embed of the session follows below:

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

  • How search can unlock the power of big data
  • How social discovery is transforming entertainment
  • Social media in Q1: commerce and discovery dominated


GigaOM