Amazon AWS SysOps – Security and Compliance for SysOps part 3

6. Guard Duty

So guard duty is a very special service that’s kind of hard to understand because we don’t have to do much. But it is an intelligent threat discovery, basically meant to protect your AWS accounts. That means that it’s going to run some analysis in the background. You don’t have to do anything. It will use the logs that’s available to it and it will just make sure that it’s protecting you against malicious usage. So it will use a machine learning algorithm, animal detection, and third party data, mingle all this together and tell you what’s up. So to enable it, you just do one click and you get a 30 days trial. You don’t want to install any software, do anything, but then after these 30 days, you start paying. And it’s not cheap. Now, input data that is going into guard duty includes cloud trail logs.

So basically it will look for unusual API calls or unauthorized deployments, that kind of stuff. It will also look at Vpc flow logs to detect some unusual Internet traffic or unusual IP addresses that are appearing within your network. It will also look at the DNS logs to look for compromise easy to instances that are starting to send encoded data within DNS queries. So it’s going to look for a lot of patterns and a lot of things that you don’t want to take care of, that Amazon will take care of for you by analyzing all your network logs, basically, and make sure that if anything is wrong, it’s going to notify you right away. On top of it, you can integrate this with a twist lambda if you wanted to automate much of this infrastructure.

So let’s have a quick look at guard duty in the UI, but there’s nothing much to do. All right, so in the cancel console, just put guard duty in and you’re taken straight to the page. So, as we can see, guard duty is to do intelligent threat detection and protect your address, account and workloads. So it’s continuous, it’s comprehensive, it’s going to allow all these events, including cloud trail and Vpc flow logs and DNS events. And then it’s going to basically give you some insights. So let’s get started. And it is so easy. First you need to basically enable guard duty, and it will get some permissions to analyze the cloud trail logs, the Vpc flow logs, and the DNS logs basically to generate findings. And then you just click on enable guard duty.

And that’s basically it. Now it’s going to go overall and do these things and analyze wise all over the time. Now, I don’t recommend that you put this all the time on because it’s going to be expensive. But as you can see on the left hand side, there’s a free trial. And right now I’m on day one of my 30 day trial. So what we get out of it is that we’ll get the number of events processed. Since you enable it. So you get how many cloudware logs, cloud trail logs, Vpc flow logs, DNS logs, and that will basically give you an estimated daily cost after the free trial ends. So if you’re very worried about the security of your Amazon’s account and you want to make sure you don’t have any people using it for malicious reasons, then this is kind of a nice thing to have.

So as you can see, you can set up the settings and see the raw permissions, you can set up some cloud watch events, you can generate simple findings to basically understand the kind of findings that guard duty generates. So let’s have a look in a second. We can also suspend it basically to set all the stuff or disable it so we don’t pay for it anymore. If we go to list, we can see the trusted IP list and the threat list, basically if you know some IP addresses, et cetera, et cetera. And then for the accounts, basically the accounts that are sharing findings with you. So let’s go back to findings. And as we can see, because we have enabled some sample findings, it will tell us what’s going on. So I say, oh, look at this. This EC two instance is mining bitcoin on your EC two? And so it says, okay, I’ve seen that, it’s mining bitcoin, it’s high severity, you should do something about it.

So it’s neat, it’s really, really neat. You get a lot of insights into stuff that can go in on other things, can be access, key ID that can be recognized, there’s some unauthorized access, maybe a brute force on your EC Two instance for RDP. So someone is trying to get into RDP by brute force, maybe there’s a Trojan on your machine. So all these things is what you can get as basically insights and findings and they’re all ranked by severity. So basically the blue is going to be just for informational, so low, then the orange is going to be for medium and the red is going to be for high severity. So that just gives you an idea of all the findings you can get out of it. And it’s kind of nice to see it once, but that’s it. It’s not something that you have to do or nothing much for you to do.

Just enable it and then over time it will give you findings and you could set up notifications. So we said you receive emails anytime one of these findings is found. So for the free trial, I’m going to obviously disable it. And as you can see, it started to already analyze my cottage logs and my Vpc flow logs in the meantime. So I’m going to accounts, sorry, settings, and then within it I’m going disable guard duty and save the settings so I don’t get to pay. So disabled it and now I’m done. So that’s it for guard duty. Just remember what it does at a high level, okay? It analyzes your internal accounts and basically try to detect threats or viruses or stuff like this. Stuff that can basically hamper take over your account or try to do malicious stuff with it, such as bitcoin mining. All right, I will see you in the next lecture.

7. Trusted Advisor

Okay, so now we’re going to visit Trusted Advisor. Trusted advisor is a service. It’s super easy, there’s nothing to install. It’s going to give you a high level AWS account assessment. So you’ll get recommendations straight away from the UI. How does it work? Well, Advisor will go ahead and analyze our AWS accounts and provide recommendations on many different topics such as customization performance, security, fault tolerance and service limits. So a popular exam question is how do we know if we’re reaching a service limit? How can we get a general overview of that? Well trusted advisor is the answer. Now the thing is with Trusted Advisor is not really free core checks and recommendations, that’s free for all customers, but a lot of it is disabled unless you have a business support plan.

Also you can get weekly email notifications directly and you can enable that from the console. And so as I said, if you’re a business and enterprise customer, you get full Trusted Advisor and you can even set cloud watch alarms when you start reaching your limits. So let’s have a look at Trusted Advisor just to get a better idea of how it works. So let’s find the Trusted Advisor service and here we’re going to have a view of multiple different topics. As you can see, I told you there would be five topics that we would see. There is cost optimization, performance, security, fault tolerance and service limits. As we can see right now, the little refresh button is checking is running because it’s actually going to do some checks on my account right away.

So while this runs, let’s go into cost optimization and cost optimization. The first thing that’s surprising is that it’s absolutely not available unless we upgrade our support plan. So if you’re a corporation, for sure you would have a support plan and then you take advantage of it. But here because I don’t have a support plan, I don’t have any cost optimization checks yet. We can see what happens. Low utilization of Amazon et. Two instances. For example, if their CPU is less than 10% idol, low balancers. So basically the balancers that don’t do much underutilized EBS volumes and associated elastic IPS. So basically all the very common ways of wasting monies on AWS, you will get some information about it here, which is really really nice.

The tricky thing though is that it complements the billing features, it doesn’t supplement them. Okay, let’s look at performance. So performance is yet again locked, and locked means that you have to also upgrade your support plan. Still we’ll see what it is. So there is high utilization in Amazon EC two instances. That means that there is more than 90% CPU instance daily on your instances. So that means they’re pretty much stuck or you need to upgrade them or something’s happening. Same for EBS volumes, large number of rules in an easy to group. Basically that can lead to performance issues. But we’re talking about a very large number of security group rules, et cetera, et cetera.

So you can have a look at all these things. Security is where it’s important. So here there’s one item that needs to have my attention. Five items are good, nothing really, really bad. But for security groups, for you, we can see which security groups have unrestricted ports. So that means that for now, port 22 is unrestricted and that can be pretty, pretty bad. So the idea is that here I have the groups, all my groups here that have enabled port 22 unrestricted and that could go and act upon it. So what you see here is that it sort of overrides with config. Config was more around compliance, this is more around recommendations. So they’re a little bit different, but they check for the exact same thing, at least for security groups.

Here we can also see public snapshots, RDS, public snapshots, bucket permissions in case we have any public buckets imus and whether or not we have MFA enabled on root account. Still we get a lot more information if we do upgrade our support plan. And I invite you to have a look at all these things right here. Okay? Fault tolerance, yet another thing that we need to upgrade the support plan for. But you get information around the edge of the snapshots, you get some availability, load and balanced to make sure you’re really balanced across all the AZ optimization for your load balancer, configuration, et cetera, et cetera.

So come some really, really good recommendations and finally importantly, service limits where we can check how we are doing for each service and we’re forgetting close to the limits. If we’re at 80% of the service limits, we’ll get a little warning and we’ll know about it. So let’s have a look at for example, EC two on demand instances. I’ll go right here and say okay, for now I don’t have any of my limits and I could click here and see all the limits as well. So, really helpful I think to have all these things. And the exam will ask you about service limits, that’s for sure. So know that Trusted Advisor is the way to get alarms and stuff like this. And then finally super important, somehow it comes up. You do have your preferences where you can disable Trusted Advisor if you wanted to.

But more importantly, you can set weekly email notification preferences and you can set email address for billing, contacts, operations, contact, security contacts. The idea is that if you want to just receive weekly emails around the recommendations you can do to improve your cost, performance, security, fault tolerance and service limits, this would be the way. And that is also an exam question. So it’s to enable these weekly email notification preferences, you have to do this through the UI. So that’s it for trusted advisor? It’s actually a very simple service, but a very helpful and you leverage it fully when you have an actual support plan for your AWS accounts. All right, that’s it. I will see you in the next lecture.

8. Encryption 101

So first an overview of encryption mechanism and the first one is going to be encryption in flights. Then why would we want even encryption in flights? Well, we want encryption in flight. Because if I send a very sensitive secret, for example, my credit card to a server to make a payment online, I want to make sure that no one else on the way where my network packet is going to travel can see my credit card number. And so I want to make sure that when I’m make a payment online, I have that green lock, I have that Https website, which guarantees me that it is an SSL enabled website and I will get encryption in flight. And so when you have encryption in flight, the data will be encrypted before I send it and then the server will be decrypting it after receiving it.

But only myself and the server know how to do these things. Now the SSL certificates are what’s going to help with the encryption. And so another way to see it is Https. So anytime we’ve been dealing with an Amazon service and it had an Https endpoint that guaranteed us that it was encryption in flight. And now the whole web, almost the whole web needs to run on SSL and Https. Basically when you have this enabled, you’re protected against the man in the middle attack. And so this guarantees that when you have that green lock and that the service certificate is valid, that no one can retrieve your sensitive information. So let’s do a quick example here is us, and we want to talk to an Https website on AWS. Could be DynamoDB, it could be whatever you want.

And then what we’re going to do is that we’re going to have to drive a super secret data, we’re going to encrypt it with SSL encryption and send this over the network. And then the website will receive that data and know how to decrypt it. Okay, very, very simple, the idea of it. But the execution is not as easy. So this is how much it’ll give you. The good news is that all programming languages know how to do SSL encryption and decryption. And all the libraries do this for you, so you don’t have to worry about anything. This is not something you have to deal with directly. The second thing is going to be called server side encryption at rest. And so that is when the data is encrypted after being received by the server.

So before that the server was receiving data, decrypting it, and using it in its decrypted form here, the server is going to store the data on its disk. And so we need to know that the server is storing the data in an encrypted form because in case the server gets hijacked by someone else, we don’t want that someone else to be able to decrypt the data. And so the data will be decrypted before being sent back to our client. So thanks to a key, usually called the data key, then that data is going to be stored in an encrypted form, and the encryption and decryption keys must be managed somewhere, usually called a Kms or key management service. And the server must have the right to talk to that key management service.

So here’s our object, and we’re going to transfer it, for example, to EBS. So it’s going to be transferred over whatever mechanism, and EBS will use a data key, and using the data key, it will perform encryption of that data. And now it’s stored in an encrypted form. And then the day we need to retrieve that data, for whatever reason, then EBS, the Able service, will do decryption for us using the data key again. And we’ll get the encrypted data and back to us over Http or GPS, for example. So this is how server side encryption works. And as you can see, the server side itself, or the service manages the encryption and the decryption and uses a data key it has access to.

So this is for server side encryption at rest, and we’ve seen that many AWS services do use that encryption at rest. Now let’s talk about client side encryption. And in client side encryption, the data will be encrypted by the client and the client is us, and the server will never be able to decrypt the data. The data will then be decrypted by a receiving client. So all in all, the data is just stored on the server, but the server doesn’t know what the data means. And the server, as best practice, should never be able to decrypt the data anyway. And for this, we could leverage something called envelope encryption. But I have a whole lecture on this later on because this is pretty advanced, but the exam will ask you about envelope encryption.

So for now, let’s just do an abstraction of it. And so we have our object, and on our client, we’re going to use a data key and we’re going to encrypt our data client Side okay, so we perform encryption with that data key. Now we send that data to any store of data we want. Could be FTP, it could be S Three, could be whatever you want, really. You put your data wherever you want in Amazon or somewhere else, and then when you receive it, your client will receive an encrypted object. And if it has access to the data key, if it can manage to retrieve the data key from somewhere, then it will be able to perform a decryption and get the decrypted objects as a result.

So, as you can see now, the encryption happens client side, okay, the server, the data store, does not know how to decrypt or encrypt the data. It just receives encrypted data. And so that’s quite secure as well. So these are the three kinds of encryption you can get yet overall except envelope encryption that will show you later on. So this is not using Edit me kms just yet. This is just an abstraction of how encryption works. I know this may be a little bit simplified, but hopefully that clears up what encryption it is. And in the next lecture, we’re going to do a deep dive into Kms.