Much has been written about the “real-time” web. Yet, depending on who you talk to, it either happened years ago or is being built right now. Fundamentally the real-time web is about giving users up-to-date information as fast as you can. Which means as technology improves real-time moves closer to being actually real time. But all of this is about the users — and driven by their demands.
Facebook and Twitter and other modern social networks are classic examples of this phenomenon. In the early 2000s you would be hard-pressed to find a service delivering real-time updates to users. This is a time when eBay looked like this:
But these days, users expect sites to deliver information as it happens. Modern web apps answer to this desire using an wide array of technologies such as push notifications and webhooks. Real-time technologies are the backbone of modern day web applications.
If you consider the real-time web to have already happened, this is likely what you’re imagining. But there is another story to be told and it’s unfolding as we speak.
The integrated web
The web is becoming more fragmented, for the better. Individual companies are building specialized products to nail a particular use-case. Consider Dropbox and Box for file sharing. Or Campfire and Hipchat for team chat. This specialization encourages competition which results in a better end product for all users. Again, it’s all about the users.
While individual companies have adopted real-time technologies to answer user demand for their own products, integration between these companies is still largely in the “Web 1.0″ era: slow and manual. Fortunately, the future is in sight.
Imagine for a moment that you’re starting a company to build the next hot project management tool.
You build an early-stage prototype of the product and put it in some users’ hands. They love it. They say your interface is so much simpler and easier to use than what they’re using now. The only problem? They can’t use it side-by-side with their current systems.
These days users expect tighter integrations with other tools they already use. Just like the real-time web did for consumer apps, it’s doing for integrations between those apps. In fact, it’s quickly becoming a selling point. Users want their bug tracking tied in, their time tracking tied in, their calendaring tied in. Not only do they expect this integration to happen, but they expect it to happen in real-time just like the rest of the web services they use.
So, you go looking for a REST API on all these other tools the customers say they want integrated into your project management app, and in today’s era you’ll likely find one. Sadly though, the only realistic way to keep your product in sync with these other tools is by polling. In other words, constantly asking each other tool’s API if anything new has happened yet. And given API limits, you’ll only be able to poll every few minutes. Your app will always lag behind changes. Yet, slow syncing is better than no syncing. The only way to give users what they want is by being slow.
The rise of webhooks and REST Hooks
To combat this, the last few years have seen a dramatic rise in webhooks. Webhooks enable one server to push data to another on demand instead of needing to constantly pull. Early on the real-time integration landscape was a fragmented with webhooks, long polling, websockets, and Comet. But recently webhooks have become the clear winner by adoption.
Webhooks weren’t the final answer, though. A critical classic flaw of webhooks is they require the user to do something. Specifically, users have to dig through complex settings and copy and paste a URL from one app into the other to make the real-time integration happen. Users hate that.
More recently, REST Hooks have emerged as a common pattern to combat this critical user experience flaw. With REST Hooks, developers set up and tear down webhook subscriptions behind the scenes so that users don’t need to do anything. These subscriptions are managed using the same standard REST API developers were using before to poll. REST Hooks solve both the real-time and user-experience problem.
The same momentum that drove the explosion of APIs is now driving the explosion of real-time APIs. Users want not just information on demand, but integration that’s always up to date. The future of the real-time web is one where all data is updated in real-time, not just your product but the integrations between your product and others.
In the end, it’s all about the users.
Mike Knoop is part-builder, part-hacker, part-designer. He dropped out of grad school to co-found Zapier, a company that integrates with hundreds of APIs across the web and is spearheading REST Hooks mentioned in this post. Follow him on Twitter.
Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.
- AWS Storage Gateway jolts cloud-storage ecosystem
- Cloud and data first-quarter 2013: analysis and outlook
- Data markets: in search of new business models