Cloud is complex—deal with it

If you are looking to cloud computing to simplify your IT environment, I’m afraid I have bad news for you.

Yeah, you might find yourself having to worry less about infrastructure, less about how storage systems work or what networking to use to connect a virtualized resource pool, or even what middleware settings are optimal for your applications. However, for every problem eliminated by choosing cloud, you’ll find it just creates more of the problems you remain accountable for—and may even create some new problems that you never had to face before.

Which is as it should be. Let me explain.

When I describe cloud computing as an application-centric operations model, one of the first questions that should come to mind is “operations of what, exactly?” Just because the cloud is focused on the application, it by no means implies that the application is all that is being operated. In fact, just as in any computing technology since the earliest electronic computers, the application can’t exist without myriad things supporting it.

And the world doesn’t consist of a single applications, but, in fact, millions of applications. Most of these are interconnected in some way, and the matrix of code, data, infrastructure, people, policies, requirements and so on that makes up modern IT is ultimately a very interconnected, complex system. Cloud computing is just one (very effective) way of dealing with that complexity.

Cloud as a complex system

What’s interesting is that it turns out science has a whole body of work around complex systems. A complex system, according to Wikipedia, is “a system composed of interconnected parts that as a whole exhibit one or more properties (behavior among the possible properties) not obvious from the properties of the individual parts.”

That’s certainly true of the modern interconnected IT environment. Just look at automated trading systems and the famous “flash crash” for an example—systems designed for increasing market returns reacted to each other in a way that temporarily crashed that very market. Other examples abound, and I’m sure your own IT environment often behaves in ways that no single application or other element was designed to do explicitly.

What science teaches us about complex systems is that they are made up of many individual agents, each of which effect and are affected by agents around them. The feedback loops of events created by agents affecting each other both directly and indirectly, combined with the mechanisms that choose behaviors to in response to those events, combine to create the systemic behavior that is so unpredictable.

Cloud as an adaptive system

The thing is, however, a certain class of complex systems, complex adaptive systems, have the additional trait that they can change their behavior in response to the success or failure of previous behaviors when a given event occurs—or when a certain series of events occurs. This ability to “learn” and adapt to the surrounding system environment creates amazing outcomes, including many of the most rich, enduring and powerful systems in our universe.

Think biology. Think economics. Think ecosystems.

IT is adaptive, in that winning functionality survives and thrives, while losing functionality dies out and disappears. Thus, those investing in building IT technologies are constantly seeking ways for their technology to survive in a changing, often hostile environment.

If an application, or function or even just a line of code fails to add value to the environment—or worse, negatively disrupts the value of the environment—it will be removed or changed, one way or another. Those that rely on IT are constantly seeking ways to optimize applications, data and technologies to take the most advantage of their systems environments.

The result is constant innovation, and constant adjustment to our needs as businesses and individuals. It ain’t always pretty, as they say, but so far it has been quite effective. (I should note that this even applies to infrequently modified “legacy” applications; there is an ongoing decision to not modify such an application, and thus it continues to survive.)

The developer as DNA

I want to leave you with one last thought, however. One of the things about complex adaptive systems is the learning or adapting traits of the agents in the system. In the world of evolution, the main agent of learning or change is DNA. In the world of IT, the agent of learning or change is the engineer or software developer.

If something goes wrong with an application, developers are on the hook to fix it, change it or kill it. If existing hardware fails to create new opportunities to innovate, engineers find new approaches to introduce into the ecosystem to shake things up.

However, developers and engineers can only make those changes one, or a few, components at a time. Nobody can configure the “system” to work an expected way. All you can do is constantly monitor the success and effectiveness of the technologies you deploy into the cloud, and constantly tweak them to make them as useful as they can be in that environment.

It’s up to people to make technologies that survive cloud as a complex system—one component at a time. That’s, well, how you deal with it.

Image courtesy of Flickr user gruntzooki.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.