Traditional enterprise data sources — be they business systems or even the exhaust from corporate websites — represent the data that is typically captured by an enterprise for analytics and business insight. However, in the new world of APIs and the app economy, organizations no longer own, much less control, all the data they need to make accurate business decisions.
For over 20 years, I’ve led technical strategy and product initiatives for databases, information integration, analytics and big data. Today I work at Apigee, where we help organizations embrace the exploding app economy built on mobile apps, defined by APIs and powered by massive streams of data. I can say that few businesses are prepared to effectively use the new sources of valuable enterprise data that is being generated “outside” the enterprise today in the app economy.
A growing number of businesses are successfully building new channels through APIs and third-party applications that tap their data and Web services. As a result, all kinds of important customer interaction is happening in apps written by other people (partners and developers), far away from the enterprise core. There are three ensuing new sources of data that organizations must be able to capture, measure and analyze to get a complete view of their customers and businesses:
- The first is the data that is generated around the use of the APIs exposed by that enterprise. This data reflects API calls, correlating Web traffic and contextual data that adds color to the API traffic.
- The second is the data that is generated by the applications that make calls to the enterprise APIs. These applications also make calls to various backend-as-a-service APIs, creating performance- and behavior-related data that reflects user and app behavior.
- The third is relevant, contextual data generated by the use of other enterprises’ APIs — such as Github, Twitter, Stackexchange — all of which annotate and add color to the specific interactions with the enterprise in question.
These three new categories of data have important characteristics that make them very different from the traditional enterprise data. For instance, traditional enterprise sources have well-built structures that enable the data sources, and the analytical processes around them, to be well modeled, enabling ETL (Extract, Transform and Load) and warehousing efforts to become mainstream. With the three new types of data, ETL and warehousing is dead on arrival. Since data changes continuously, its shape is evanescent, and the analytical needs change day by day.
This loss of control requires new data marketplaces and data syndication models that few enterprises are currently prepared for, but there are a few important steps organizations can take today to start.
Step one is to have an API so you can exercise control over how your partners and third-party developers access your data. This seems obvious, but you’d be shocked to learn how many companies are getting their websites scraped and have no control over how their data is getting used.
Step two is to surround your APIs with other services (such as user management, state management, event handling, etc.) that either you provide or that you can access through relationships with the providers of those services. This will help ensure the success of the apps built on your APIs — and make it easier for you to directly capture or infer the usage of those apps.
Step three is to determine the value of other people’s APIs and the data that sits behind those APIs. Just like you are exposing your APIs and making your data and services available, other parties in the app economy ecosystem are doing the same. The business strategy around this quid pro quo is something organizations will have to master.
Finally, having access to the data (your APIs, the apps built around your APIs, and other third parties) is still just that — data. You have to stitch all of this data together to create powerful signals. A good thing about this new data is that stitching is actually easier, not tougher, than in core enterprise systems. Data is more verbose, information is meant to be shared and hence better described, and more data creates opportunities for better stitching.
Business leaders should be concerned that they don’t have full visibility into how customers are using their services in the new channels created in the app economy. The truth is, most of the analytics and data and insights you get today are wrong. You simply don’t own all the data anymore. Every enterprise has to rethink their data platform for this new world. Enterprises that can capture and analyze new big data outside of the enterprise will succeed in the app economy, those that do not, will fail.
Dr. Anant Jhingran is vice president of products at Apigee. He joined Apigee from IBM, where he was vice president and CTO of IBM’s Information Management Division and co-chair of the IBM-wide Cloud Computing Architecture Board. At IBM, he was responsible for the technical strategy for databases, information integration, analytics and big data, and he helped deliver IBM’s PaaS capabilities.
Jhingran will discuss the Cambrian explosion of APIs at GigaOM’s Structure Europe conference next week in Amsterdam.
Image courtesy of Flickr user jeff_golden.
