History shows that the most successful products and services — from the VHS video recorder to the Apple iPad — are the ones that are the most user-centric, and in order to construct those kinds of experiences companies need to use all the data that is at their disposal, WiBiData co-founder and chief technology officer Aaron Kimball told the Structure:Data conference in New York on Thursday.
Image courtesy of Cory Doctorow via Flickr
The problem for many companies is that information about their users is spread across hundreds or even thousands of different fields in various databases, and it’s difficult to put it all together in real time. But doing that successfully is becoming more and more important as services become more personalized.
Kimball, whose company has been funded by Google chairman Eric Schmidt and Ron Conway’s SV Angel group, said that WiBiData’s solution is to put together a platform that combines big-data aggregation and analysis tools like Hadoop and Hbase and Apache’s Avro, which can pull in and make sense of data coming from multiple databases in one location, and then make it easy for companies to extract and make use of that information in real time. In addition to tools that combine those services, WiBiData also has a user interface and dashboard that allow companies to pick and choose which data to use quickly.
Online users in particular have grown used to services like Amazon and Netflix, which provide real-time recommendations based on what a user has done in the past or other elements of their user profile, Kimball said. This not only saves a user time, but exposes them to items they may not have even known they wanted to search for — and thus creates a better user experience.
So if your company is selling hardware and a user puts a hammer in their online shopping cart, you want to be able to suggest that they buy some nails as well. But this has to be done in milliseconds if it is to be useful, and that requires manipulating large amounts of data in real time to update or alter the user’s profile and pull in the new selection.
“We are now capable of collecting and analyzing orders of magnitude more information than at any other time in history,” thanks to the database tools we have, Kimball said, but “the one constrained resource is understanding.” Users of online services now expect to have things personalized in real time, and if your company can’t provide that then you are going to be behind the curve, he said.
Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.
- A near-term outlook for big data
- Dissecting the data: 5 issues for our digital future
- Defining Hadoop: the Players, Technologies and Challenges of 2011
