The hubbub around Tuesday’s Citrix announcement has died down a bit, but I’d like to highlight one particular area of confusion: the cavalier use of phrases like “AWS compatibility” and “Amazon-style architectures.” These terms have been used interchangeably, but are they really the same thing?
As part of the Citrix announcement, Sameer Dholakia, their general manager of cloud platforms said:
“….we believe the biggest winners in the Cloud Era will be clouds built on a platform that is designed from the ground up with a true Amazon-style architecture, proven at scale in real production clouds, compatible with the Amazon architecture and fully committed to open source. With the significant momentum CloudStack has gained….it is the only cloud platform on the market that even comes close to meeting these requirements.”
Of course, this is a shot across the bows for VMware, but most pundits, bloggers, and journalists focused, somewhat rightfully, on what it will mean for OpenStack.
But what does this statement mean? How much is real, and how much is marketing-speak? What does it mean for VMware and OpenStack? What does it do for Citrix’s strategy?
Before digging into these questions, I want to make clear that I am absolutely biased. Cloudscaling, the company I cofounded, is a longstanding member of the OpenStack community, and the company’s Open Cloud OS is based on OpenStack software.
I’m also in a better position to understand the reality here. Working with KT in 2010 and 2011, Cloudscaling designed and built one of the largest CloudStack deployments to date. Working with a global top 10 carrier in 2011 and 2012, Cloudscaling helped them kick off one of the largest OpenStack deployments to date.
Parsing the quote
Citrix makes several claims:
- Winners in the “Cloud Era” will be “designed from the ground up with a true Amazon-style architecture”
- These winners are “proven at scale in real production clouds”
- The winners are also “compatible with the Amazon architecture”
- The winners are “fully committed to open source”
Many of these claims seem to focus on “Amazon-style architecture” and Citrix’s right to claim the mantle of Amazon-ness:
“[CloudStack] is the only cloud platform on the market that even comes close to meeting these requirements.”
A bold claim, but is it true? Eucalyptus would claim otherwise and I won’t be bashful in saying that I think our OpenStack-powered Open Cloud OS will be more compatible with Amazon in every way when compared to Citrix. Vanilla OpenStack can also make similar claims.
So, what is “Amazon-style” and what is “compatible”?
Amazon-style architecture
There are a number of ways to interpret “Amazon-style,” but to be meaningful, we can assume that an amount of Amazon compatibility is implied. If “Amazon-style” means inexpensive, it means nothing. Anyone can build a cheap cloud.
Fortunately, Citrix provides a way forward when they also claim “compatible with the AWS architecture,” a more clear-cut claim.
AWS compatibility
What does AWS compatibility mean? Surely it means more than ordering virtual machines on demand. If not, then every cloud is “AWS compatible.” It must mean more. AWS-style networking, storage, and compute? Similar hypervisors, perhaps, given Citrix is the owner of Xen? Perhaps, it’s just providing AWS API compatibility?
How is an “Amazon-style architecture” compatible with AWS? Well, for most, I think it’s about replicating Amazon’s architecture (or, at least, elements of it) besides the APIs.
CloudStack versus AWS
I have heard that many of today’s CloudStack deployments were influenced by the architecture Cloudscaling designed at KT. This architecture involved the following key elements:
- Scalable layer-2 hardware virtual area network (VLAN) using Arista switches
- Local, in-rack storage area network using NexentaStor
- XenServer clusters
- CloudStack as the VM control system
- Cloudscaling’s provisioning system and accompanying chef recipes
- Cloudscaling’s methodology for building hardware blueprints
- Supporting open source software for monitoring, logging and related functions
Now let’s examine these elements and related technology differences between AWS and CloudStack.
Hardware VLANs are not AWS native networking
Obviously, standard AWS doesn’t provide hardware VLANs in its architecture. The closest capability that is a cloud service provided by AWS is Virtual Private Cloud (VPC). However, it’s not their default networking but an add-on. The default AWS networking is a flat layer-3 network, which CloudStack can support, but typically does not. Much of their value-added functionality (e.g. load balancing, network area translation) disappears when using flat networking. Default CloudStack deployments aren’t network compatible with AWS. Maybe you could claim that they’re compatible with AWS VPC, but even that is a bit of a stretch (more on that below).
AWS EBS is not a SAN
AWS Elastic Compute Cloud doesn’t use SAN storage for standard VMs, although it does provide a similar type of capability in Elastic Block Storage (EBS). Instead, EC2 provides primarily direct-attached storage (DAS) that is surfaced as ephemeral storage for each VM. No guarantees are provided around the persistency of local VM storage. This is in contrast to standard CloudStack, which typically uses XenServer clusters and persistent storage to provide high availability (HA) for a VM, where a VM is guaranteed to never be lost.
Clearly, this is different from AWS, which means the storage architecture for CloudStack is different. OpenStack, in comparison, uses a default storage model which is exactly like AWS.
Xen is XenServer?
AWS doesn’t use XenServer, but has their own customized version of Xen. This is close enough that we can call it the same. If “Amazon-style” and “AWS compatible” means the Xen hypervisor, Citrix claims are good; however, anyone running Xen is now AWS compatible.
1999 called and wants its application architecture back
CloudStack is a single monolithic piece of Java code. Most of the code resides in a single .jar file and runs on a single Java app server by default. This is a common architecture — from 1999. As you might expect, AWS has much more software, mostly written in a distributed service-oriented-architecture (SOA) fashion. The core software architecture is not similar at all between AWS and CloudStack.
CloudStack’s code focuses on features desirable in smaller deployments such as HA or high-availability of VMs (a common request in VPS-type environments that is irrelevant for customers who use DevOps-style automation). Amazon doesn’t provide HA for VMs. You can boot a VM off of a persistent EBS volume, but that is about as close as you can get. Again, this is dissimilar in architecture and capability.
Advanced networking in CloudStack and AWS
Regarding advanced networking capabilities in CloudStack that are comparable to AWS EC2 and VPC, the picture is rather bleak. When deployed with hardware VLANs, the CloudStack networking model depends on providing a router-VM for every customer. This router-VM provides networking functionality such as NAT, routing, load balancing and DHCP. All router-VMs are a certain size (number of cores, amount of RAM, etc.). The size of the router-VM directly impacts amount of throughput and number of TCP sessions possible through the load balancing software.
At KT, we did a significant amount of tuning of the router-VMs to support bigger deployments. This is problematic in that if all router-VMs are upsized, to say 2GB RAM with 2vCores, you give up significant capacity. If downsized to something smaller, they are unable to provide acceptable performance. It’s impossible to right-size these router-VMs or to scale them dynamically.
Regardless, this is nothing like Amazon. Amazon uses edge networking services that are horizontally scalable and are multi-tenanted rather than one router VM per customer.
CloudStack’s native API is not AWS compatible
The CloudStack API is not AWS compatible. Instead CloudStack provides a piece of software, called CloudBridge, that is installed co-resident with CloudStack. CloudBridge translates AWS EC2 API calls to CloudStack native API calls.
CloudBridge only provides EC2, not S3. In this regard, stacks like Eucalyptus and OpenStack provide more AWS API capability by providing both EC2 and S3 APIs. How then is CloudStack “the only cloud platform on the market,” “designed from the ground up with a true Amazon-style architecture” and “AWS compatibility”?
There are additional issues with Citrix’s claims that CloudStack was designed from the ground up with an Amazon-style architecture. It’s not even close nor can it be without a ground-up re-write.
Where does CloudStack play?
Given this comparison, it doesn’t sound very good for CloudStack. Yes, CloudStack has a number of deployments and traction. It’s been very successful. However, when we see customers choosing CloudStack, they are almost never building an AWS-style open cloud. Instead, they want a cheaper VBlock.
CloudStack, if anything, is an open source attack on VCE and VMware that is largely irrelevant in the open cloud space. When talking to other OpenStack-based open cloud vendors, they tell me that CloudStack is rarely competing against their products except when customers don’t understand the difference between an enterprise-style cloud and an open cloud.
If you want a cheaper VCE VBlock, I think CloudStack is a great option. If you want an open cloud that is AWS compatible for both architecture and APIs, CloudStack is a terrible option. OpenStack is a true open cloud solution. It has more architectural affinity with AWS than CloudStack.
Bringing reality to the unreal
No one should be under the illusion that CloudStack is more AWS/Amazon compatible than any other open source cloud software. Eucalyptus and OpenStack have similar or better claims to AWS compatibility.
To suggest otherwise is misleading.
Randy Bias is the co-founder and chief technology officer at Cloudscaling, which provides an OpenStack-powered open cloud solution with support for AWS and Rackspace APIs.
Image courtesy of Flickr user Radio Saigón.
Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.
- A near-term outlook for big data
- Migrating media applications to the private cloud: best practices for businesses
- Dissecting the data: 5 issues for our digital future
