Skip to main content

Business Bonus: AWS Outage (12/7/2021)

 

There was an AWS outage a few days ago; You were probably impacted in one way or another.

text: I got an email from my washing maching saying aws-east-1 was off line so that's....a thing

If any of my appliances would ever e-mail me, I think I'd die of shock.

Every thing from Disney+ to McDonalds was affected, as US-East-1 was US-East-None for a few hours. Even Amazon delivery drivers and warehouse workers couldn't complete their breakneck, no bathroom, tasks.

My Alexa couldn't reach AWS. "Guess I better attach it to the new hotspot..." I thought, before moving on with my day and not doing that. Turns out, my hotspot wasn't the issue.


This is not the first cloud provider outage to happen, not even this year. So, how can technicians prevent this from happening?

Outages will happen, and this may have broken whatever 5 9's streak AWS strives for as a whole, but I'm 100% sure AWS networking technicians are working to implement high-level ways to recover quickly from this in the future. Someone uncovered that error, and should be commended. Who knows how long it would have been lying in wait?

That doesn't mean you can't be prepared. The streaming giant Netflix uses AWS to host its thousands of shows and movies - and they still have onsite systems and network engineers in Los Gatos, California.

What about multi-cloud?  That a thought, but this Twitter thread showed me the cons to that idea. 

I think nothing but the absolute largest companies would benefit from multi-cloud; The question is, can it be justified cost-wise. 

What would work for your company - 

Losing x$ during a 4 hour outage
Spending x$ a year to maintain two cloud relationships?

It depends partially on how often you expect your primary cloud provider to fail - Something that's happening more and more.

VentureBeat has a great article of what happened, with thoughts from many industry professionals.

Comments

Popular posts from this blog

Connecting IoT Devices to a Registration Server (Packet Tracer, Cisco)

In Packet Tracer, a demo software made by Cisco Systems. It certainly has changed a lot since 2016. It's almost an Olympic feat to even get started with it now, but it does look snazzy. This is for the new CCNA, that integrates, among other things, IoT and Automation, which I've worked on here before. Instructions here . I don't know if this is an aspect of "Let's make sure people are paying attention and not simply following blindly", or an oversight - The instructions indicate a Meraki Server, when a regular one is the working option here. I have to enable the IoT service on this server. Also, we assign the server an IPv4 address from a DHCP pool instead of giving it a static one. For something that handles our IoT business, perhaps that's safer; Getting a new IPv4 address every week or so is a minimal step against an intruder, but it is a step. There are no devices associated with this new server; In an earlier lab (not shown), I attached them to 'H

Securing Terraform and You Part 1 -- rego, Tfsec, and Terrascan

9/20: The open source version of Terraform is now  OpenTofu     Sometimes, I write articles even when things don't work. It's about showing a learning process.  Using IaC means consistency, and one thing you don't want to do is have 5 open S3 buckets on AWS that anyone on the internet can reach.  That's where tools such as Terrascan and Tfsec come in, where we can make our own policies and rules to be checked against our code before we init.  As this was contract work, I can't show you the exact code used, but I can tell you that this blog post by Cesar Rodriguez of Cloud Security Musings was quite helpful, as well as this one by Chris Ayers . The issue is using Rego; I found a cool VS Code Extension; Terrascan Rego Editor , as well as several courses on Styra Academy; Policy Authoring and Policy Essentials . The big issue was figuring out how to tell Terrascan to follow a certain policy; I made it, put it in a directory, and ran the program while in that directory

Create a Simple Network (Packet Tracer) + A Walkthrough

Again; I've done this, but now there's so many new things, I'm doing it again. The truly new portions were...everything on the right side of this diagram; The cloud needed a coax connector and a copper Ethernet connector. It's all easy to install, turn off the cloud (Weird), install the modules. Getting the Cable section of Connections was an unusual struggle - The other drop down menu had nothing within. It required going into the Ethernet options and setting the Provider Network to 'cable', which is the next step AFTER the drop-downs. The rest was typical DHCP and DNS setups, mainly on the Cisco server down there. The post is rather short - How about adding a video to it? Find out what A Record means - This site says 'Maps a name to an IP address', which is DNS. So it's another name for DNS? You can change them (presumably in a local context) to associate an IP address to another name.