Christmas Eve AWS outage stings Netflix but not Amazon Prime

Oh to be a fly on the wall for the conversations that must be going on between Netflix and Amazon engineers this holiday season.

If you’re not a Netflix subscriber, you may not yet know that issues at Amazon’s US-East data center facility took down Netflix’ streaming service on Christmas Eve — arguably the worst possible time. Starting at 1:50 p.m. PST, as GigaOM’s Janko Roettgers reported, Amazon’s US east facility reported issues with its elastic Load Balancing Service that carried over into Christmas morning.  Interestingly, Amazon Prime Instant Video streaming service, which competes head-on with Netflix and which also runs on AWS, appeared to be unaffected by the US East snafus.

The latest update to the AWS status page reads:

Dec 25, 4:36 AM PST We continue to work on resolving issues with the Elastic Load Balancing Service in the US-EAST-1 region. These issues are affecting updates to both existing and newly created ELBs. A subset of ELBs that made configuration changes or changes to registered instances during the event are experiencing errors or receiving reduced traffic. We continue to work toward a full recovery of the service. We apologize for the continued impact.

Other sites, including Heroku’s Platform as a Service, were also affected. Heroku, like Netflix, have been down this path before with previous AWS US East glitches.

herokuoutage

This, the latest of several problems at Amazon’s Ashburn, Virg. facility, highlights a couple big, recurring issues for Amazon, its partners, rivals, and customers.

1:  US-East is Amazon’s largest and oldest data center facility and often is the first  to host new services. It’s also the facility at ground zero of most of the AWS-related outages over the past few years. Still, many customers feel they have no choice but to deploy there since it’s usually the first AWS data center to host new services (e.g. Amazon’s new high-storage instance types announced last week are only available from US-East for now.) And US-East tends to be less pricey than Amazon US-West facilities in California and Oregon.

2: Working with AWS now is a lot like a software company partnering with Microsoft in the 80s and 90s — it’s both your biggest partner and your biggest rival so tread carefully.  At AWS: Reinvent last month, Amazon CEO Jeff Bezos touched on this topic of “coopetition.”  Amazon Prime Instant Video competes with Netflix but “we bust our butts every day for Netflix,” Bezos said.

3: Issues like this one can only help AWS rivals in the OpenStack community — Rackspace, Hewlett-Packard et al that are trying to position their cloud services as options with better service and support, if not the same huge scale as AWS.  Partners might also take a harder look at other infrastructure providers like SoftLayer and Joyent. Just saying.


GigaOM