Ask Mehul Nariyawala and Navneet Dalal about the Xbox Kinect, and you’re going to get some passionate responses. Nariyawala and Dalal are the co-founders of the gesture recognition startup Flutter. Their Flutter app, which makes it possible to control apps like Spotify and iTunes with a few simple gestures, gets compared to Kinect all the time. But the duo strongly believes that Microsoft got this all wrong. “What they are doing is turning your hand into a mouse,” said Nariyawala when I visited Flutter in their new San Francisco office earlier this month.
Microsoft’s Kinect, they argued, wants to train you to perform big motions, and navigating the UI of Xbox requires you to press buttons and swipe Netflix film covers across the screen: Interaction patterns that work fine on your desktop or even your iPad, but simply lead to tired arms when performed in front of a camera or optical sensor. Flutter wants to instead focus on small gestures, and make machines smart enough to recognize even minimal changes, said Nariyawala: “The analogy is not Minority Report, but R2D2.” It’s time to train the machines, not the users, he argued.
Controlling Spotify, iTunes and VLC
“The analogy is not Minority Report, but R2D2.” Flutter doesn’t want you to learn how to use your body like a mouse, but teach machines to understand you.
To be fair, Flutter isn’t exactly Star Wars smart yet. The app uses your computer’s webcam to capture gestures and then control media playback in apps like Spotify, iTunes, VLC and Quicktime.
Flutter started out with one simple gesture used to start and stop media playback, and added the ability to jump to the next or previous song with a new release that just hit the Mac App Store Sunday (a Windows version will be released on 9/30). All of that is fun, but not incredibly useful. Sure, Flutter users can control media playback from up to six feet away by simply holding up their hand. But they could just as easily walk over and hit a single button.
Still, there is something incredibly compelling about Flutter. The app captured my imagination from the first time I used it a couple of months ago, and I immediately began to wonder: What else can you do with this kind of technology?
A lot, if you believe Nariyawala and Dalal. The duo is working to extend Flutter functionality to a browser plugin to offer gesture interaction for web-based services. That is harder that it may seem: there are countless video sites and music services out there, each with a slightly different feature set. “I don’t think people want to remember hundreds of gestures,” said Nariyawala. So how will Flutter keep things simple? “We want to build a vocabulary for web applications,” explained Dalal. A few gestures that work everywhere on the web.
Making Flutter work on mobile
Gestures are just a first step. The duo showed me a few first technical demos of Flutter running on the iPhone, only to bring up an important point: Raising your hand in front of your computer is not a big deal. But gestures on the phone, while you’re in the subway? Pretty silly, and also hard to read for the phone, since your hand is likely far too close to the camera. “In mobile, facial expressions are more accurate,” explained Dalal.
The bigger vision is to give machines eyes, and make them aware of their users, he added. Why wouldn’t a computer automatically lock its screen once you move away from your desk, he asked? And why shouldn’t your phone recognize whether you’re smiling or having a bad day?
Those are some very ambitious goals, and Flutter is taking its first steps to get there. The company, with a total headcount of seven, has some serious image recognition talent. Dalal, whose computer vision experience goes back to the 1990s, told me that the team worked three months on dealing with false positives alone.
It also spent some serious time on a small app footprint: The current Flutter executable is just 2MB, which makes it a prime candidate to be included in other apps. The founders told me that they eventually see Flutter powering gestures across a variety of apps and web services on PCs, mobile devices and even connected TVs. But for now, they’re concentrating on their own Flutter app to crack the code for the perfect gesture vocabulary – which definitely won’t involve any theatrical, Kinect-like arm waving.
Flutter raised a total of $ 1.4 million in funding from Andreessen Horowitz, NEA, Spring Ventures and others.
Robot eye image courtesy of Flickr user dwstucke. R2D2 image courtesy of Flickr user Roger Schultz.
