Amazon AWS SysOps – Security and Compliance for SysOps part 2

4. AWS Inspector

Okay, so now let’s talk about AWS Inspector. So this is only for easy two instances and that is very important. The exam will trick you into saying could you use Inspector on RDS? The answer is no, you cannot. The only way you can use Inspector is on EC two instances. So what does Inspector do? Well, it helps you analyze the known vulnerabilities or the unintended network accessibility on your EC two instances only. Why? Because you need to install an Inspector agent and you need to install this on the OS operating system of your EC two instances. Then you define a template. We’ll see this in the hands on for the rules package, the duration, the attributes and SNS topic. There is no custom rules possible.

You can only use what Amazon defines to be a rule for the Inspector. So you really are left into Amazon’s hand for this. And then after the assessment, you’ll get a report with all the list of vulnerabilities that have been found on your EC two instance. So what does Inspector evaluate? Well, this is only for EC two instance. Remember this. And for network assessment it’ll just look at network credibility and then for host assessment. So on the machine itself, the OS, it’ll look for common vulnerabilities and exposures. It will look for CIS to center for Internet security benchmarks, it will do security best practices, runtime behavior analysis and so on. So how about we just go straight in and try it out.

Remember, Inspector equals easy to instances from within. We have to install an agent on it. So let’s go ahead and get started. Okay, so let’s get started with Inspector. So I’ll just step in Inspector and we are taken straight to the Inspector console. Now, as you can see, the Inspector console will tell us to install the Aos agent on the EC two instance. Then we’ll run the assessment on the assessment target and finally we’ll analyze the findings. So, I’m going to get started. And remember, this is only for easy to instances. Now you have welcome to Inspector and basically you can just run and weekly or run once or do an advanced setup for this console. But what I just want to do right now is just do cancel to be taken straight to this dashboard.

Oh, by the way, I’m just going to go to my region which is going to be Ireland. And as you can see, Inspector is not available in all the regions. So make sure you choose the region where you have Inspector available. So I’m going to choose EU Ireland, and then I’m going to basically create an assessment. So we need to define assessment targets and then we need to define assessment templates and assessment runs. Okay? So the first thing we have to do is to go to EC two and install the agent on the EC two instance. So I go into instances, launch an instance and I’m going to choose maybe Amazon Linux AMI 2018 I’m not choosing Amazon Linux Two just to have something a bit older so I’m sorting this one okay, this is great.

I’ll use a T two micro then I’ll configure the instant details everything looks fine add the storage this looks fine too add tags and this is very important I’ll name this instance my instance and then I’ll just add a tag and you can have whatever you want but I’ll just say Inspector True. Just show that because Inspector will use tags we’ll need to just say Inspector True so that my instance is recognized by Inspector itself you don’t have to name this Inspector but it’s just nice to name it the same way as a service. Then we configure the security group and we can create a new one we’ll allow Ssh from anywhere, we’ll also allow Http from anywhere and maybe this is fine then click on Review and Launch and click on Launch and now we just choose the existing key pairs that we have and we’re ready.

Our instance is getting started okay, so now I’ll have to wait for it to get started and then I’ll Ssh into it. So now I’m going to Ssh into my instance and here I am in it. Now, there are two ways of installing the agents. Either we type install inspector agent on EC two, and then you will find documentation how to install the agent. And that is fine. You can look at it, but it’s just you have to do a Webget or a curl, and then you do Sudo bash, install UFOs, and then you’re fine. Or the really cool way you can do it is automating it using SSM run command. So for this we’ll go to assessment targets and here we’re going to create just a bunch of targets. So as we can see here, this targets all instances, all rules.

But maybe we’re just going to create our own targets and we’ll call it Inspector instances. And basically instead of taking include all easy to instances in this region and account, we will just say, okay, we’re going to use the tags Inspector true. Which is nice, because now we can say which instances are going to be operated by inspector. And there we can say, install the inspector agent on all the easy two instances in this assessment target. And for this, we need to have the SSM agent installed and an IAM role that allows to do run commands. So let’s change the IAM role right here and if you do remember in the beginning of this course we did create an IAM role basically to do SSM stuff.

I call this Amazon EC two role for SSM. So we’re going to apply this role right now to this instance and now this instance should be able to issue and to be used by SSM. Okay, so we’re fine here. Now I’m going to preview this and as you can see we found one instance that has been matched by our assessment targets because we specified the tag Inspector True. As you can find right here, the tag is Inspector True. Excellent. And I’m going to save this now it says, okay, the run command has been successfully issued to install the Amazon respected agent and we can basically check the agent status by choosing the preview target button in the SSM run command as well.

So let’s have a look. Here we can do preview targets and here it says Agent status unknown. So we have to wait until this run command is finished. But what we can do as well is go straight into SSM. So we’ll do SSM and go to Systems Manager. And here we’re going to see again this run command thing. So we’re going to run command and command history. And as you can see right now, the Amazon Inspector managed a Tos agent was run and unfortunately it was not run on the right number of targets, zero targets. So we have to do it again. Let’s do it again. So install agents who run command press OK, it is possible that the agent itself was not installed for SSM. So let’s have a look and see if that works.

What we can do is go to manage instances and in there we can see that yes, our instance is online so there’s no reason why it wouldn’t be associated out. The reason is that it’s not getting installed and it’s good. I’ll just keep this in the video because it’s good troubleshooting to do the cups is that the association status is pending so the instance is still getting registered with SSM. When it’s fully registered then we’ll be able to reshoot the run command. So let me just wait a little bit. Okay, so now my association status is success. So if I go back to Inspector and install the agent using the run command, hopefully this time this will work. So let’s go back to our Systems manager and go to run command.

Here, this one is in progress and now we have one target. So that’s perfect. Our instance has been identified as a target and now we’re just going to wait. It’s done. Command history, it was a success, there was no errors. So now our instance has automatically the Inspector agent install on it, which was a cool way of doing it. We could have done it manually but we just did it using SSM and run command which is really, really nice. Okay, next run templates. We can use the default one but I’m just going to create one just to show you how it works. We’ll just say my demo template and the target instances is going to be the Inspector instances.

The rules package is whatever analysis you want to do on it. So I’ll just select the five analysis right here and you can see there’s a version in case AWS includes more stuff into the rules and the duration is going to be 1 hour. We’re not going to use an SNS topic and if we wanted to, we could use tags as well to just say okay, Inspector True and this is great. Now assessment schedule, we can set up it to be recurring to occur every seven days but I’m just going to say no, I’m fine just with one and I’m going to say create and run. And here we go, the assessment run has started so now all runs is one of it and if you go to assessment runs on the left hand side, as we can see it’s starting to collect data and it’s going to run it on my instances.

So now I have to wait for 1 hour until we basically get some insights but you could show the agent by doing on the bottom left to show any of those agents and as we can see my agent is healthy and it’s ready and it’s working and it’s installed on my instance so that’s perfect. And we could show the status basically to see how long this has been running for. So this has been running for 26 seconds right now so I’ll just wait for 1 hour and get back to you. Okay, so my analysis is now complete and this analysis found 117 findings and so we could click on this and get some information. By the way, if you do show status, it shows you how many telemetry message has been sent from the agent.

So about over 9000 in 1 hour. So it’s pretty interesting and then if we go to the EC two instance we could see that the CPU usage has been going on for about an hour which is quite interesting as well. That means that the agent itself does use a little bit of CPU and does have an impact on your instances. A little one, but still an impact. So it’s good to know. Okay, so if we go to the findings now and we can just see all of them, we can filter them by severity. So there’s high, medium, low and informational. So you could literally look at this one for example, and it says that for example, for this one it is a common vulnerability and exposure is 1. 1. And so it says that my instance is vulnerable to CVE 2018 11236.

And so it just tells you what’s going on and it tells you the recommendation. Maybe you should update your OS package and run it again and then it will go away. So it’s very interesting and I think it’s quite nice to run this every week to get information around your instances OS patching and get some information. Basically the exam will ask you hey, we need to understand if our instances are vulnerable to certain attacks or if they need to be patched. Then the answer is use Inspector to see if they need to be patched. But then to patch them you could use SSM, for example. So you can also download a report and you can be a Findings report or a full report. And you can choose HTML or PDF.  

So I’ll just choose PDF and click on Generate report. And from this you can have this report and it will be open when it’s ready. And the reason you would have these reports is maybe to keep them into a bucket or something like this. And then you would just show that every week you do look at these long reports that can be used for audits or whatever. As we can see, this report is 951 pages, so it’s quite long and it tells you what is being tested, et cetera, et cetera, which can help for audit and compliance. So pretty interesting, I think, overall, I think it gives you a good overview of what Inspector does. And then when you’re done with Inspector, you can take your ECG instance and you can just terminate it. That’s it. That’s all for inspector. I will see you in the next lecture.

5. Logging in AWS

So just to help you for the exam and to make you understand what kind of logging is available in AWS, here’s a short lecture. So basically, if you want to have compliance requirements, there’s many services that AWS provide logs for. It could be security logs or audit logs. So service logs will include Cloud Trail and here we can trace all the API calls and we’ve done this as a handson so we know how that works, config rules where we can track the config and the comp plans over time. And we’ve seen this as well in the past section. Cloud Watch logs if we want to have full data retention. So for example, we want to log application logs or whatever, we can do it there vpc flow logs, which is to view the IP traffic within your Vpc.

Now we haven’t seen Vpc flow logs in detail just yet, but we’ll do it in the future list section and we’ll see how they work. There will be Elv access logs for your load balancers and they will give you the metadata of requests made to your load balancer. And we’ve had the chance to look at it cloud Front logs, which is to basically look at the logs coming straight from Cloud Front, your web distribution against some metadata of access and it’s going to give you the access logs, web application firewalls logs. So if you enable Waf, then you get full logging of all the requests analyzed by the service, which is really, really nice. And the cool thing is that all these logs you can put them in is three and then you can analyze them using AWS Athena. And so that is a very common exam question.

They will say, oh, we have this log, how we can analyze it? How can we quickly know or explore what happened to our Elb, even though maybe our E two instances were terminated and we lost the logs on their machines. Well, we can use Athena plus Elb access logs plus S Three and that’s the combination. So just remember that a lot of services, and we’ve seen them in this course, do provide logs. They are able to put these logs into S three and then we’ve seen how to analyze these logs into S three using Athena. So this is the idea. If you Google Athena analyze Cloud Front logs, you’ll get the query right away, same for Elb, same for Cloud Trail, et cetera, et cetera. Now also you should know that if you do put all these audit and security logs and compliance logs and you put them in S three, it is great to encrypt these logs.

And then for the bucket where you put all these logs, you can control the access using IAM and bucket policies and even manufacturer authentication. Finally, if you need to retain these logs for a very long time, remember you need to move these lugs to Glacier for cost saving or if you enable Glacier vote Lock, then as we’ve seen, then we get compliance and saying no one can touch these logs for maybe seven years or whatever. If you’re more interested into basically logging and security in AWS, there is a white paper you can read, which is quite interesting. But this is enough for you to understand basically the scope of all this logging that exists in AWS, how we can analyze it, how we should store it, and how we should have cost saving and compliance on top of it. Okay, that’s it for this theory lecture. I will see you in the next one.