San Francisco startup BitDeli launched on Thursday with a simple mission: To let as many people as possible measure as many things as they want to. If it’s correct, the secret to the company’s success will be just a few lines of Python code.
BitDeli is targeting small to medium-sized businesses that need something between the easy-but-limited Google Analytics on one end of the spectrum, and a full-fledged Hadoop cluster on the other. According to Co-founder and CEO Ville Tuulos, many companies start small with some simple web analytics, but eventually realize they need something more. Especially if that something more involves tracking custom metrics unique to their business (e.g., how many Mac users are buying T-shirts or how many people are buying products within social games), he said, the natural instinct is to do what they see everyone else doing and try deploying Hadoop or maybe a MongoDB database.
The problems with that approach are pretty easy to spot: Deploying and managing those types of systems can be difficult, and they’re not particularly helpful if you’re trying to track your data in real time. BitDeli fills the gap, Tuulos explained, by letting developers easily set up custom analytic functions using Python scripts. They can be as simple or complex as necesary — even incorporating techniques such as machine learning — and they’re consumable by the entire company via a simple collection of dashboards called “cards.”
The service, which is hosted on the Amazon Web Services cloud, also keeps a database for each user so they can analyze historical data as well as the streaming data that BitDeli really focuses on.
Tuulos compares what BitDeli is trying to do to how frameworks such as Rails democratized web development earlier this century. Development used to be a heavy and expensive process, but open, lightweight frameworks made it so “10 times more developers can do things,” he said. He analogizes Hadoop and MongoDB to those first-generation development methods, while BitDeli is the lightweight new tool for the masses.
Sure, Tuulos acknowledges, there are workloads for which Hadoop is definitely still the right solution, but if you were to start thinking today about what most companies need to fulfill their analytic requirements, “you don’t come up with Hadoop,” he said. Tuulos ought to know — his previous gig was working on Nokia’s data team alongside its petabyte-scale Hadoop cluster. But, he said, when the data team are the only guys who know how to interact with the system in order to run jobs, that team becomes a bottleneck.
With BiDeli, he explained, scripts can be written and analyzing data in production in minutes, and developers can easily iterate on them when they need to be tweaked.
Whatever tools people choose to use, though, Tuulos’s analogy to web development is hard to argue against. Other companies, including Structure 2012 Launchpad winner Keen.io, are doing very similar things for mobile analytics, and even developer-focused Hadoop frameworks such as Continuuity and Mortar Data are trying to make it fast and easy to analyze big data. Analytics has to become more of a continuous development situation than a waterfall situation, Tuulos said, and “you have to increase the number of people capable of doing these things themselves.”
Feature image courtesy of Flickr user ramcreations.
 |
 |