Amazon AWS SysOps – S3 Fundamentals Part 2

4. S3 Encryption

Okay, so now let’s talk about Amazon S Three encryption for your objects. The idea is that you upload objects onto Amazon S Three, and these are servers of AWS. And so you may want to make sure that these objects are not accessible. For example, if someone gets into the Amazon servers, or you want to make sure you are dear to some security standards set up by your company. So, as such, Amazon gives you four methods to encrypt objects in Amazon Sate. The first one is called SSCs three. This is to encrypt Seas using keys handled and managed by AWS. The second one is Sedums to leverage AWS key management service to manage your encryption keys. The third one is SSCC, where you manage your own encryption keys. And finally client side Encryption. Now, we’re going to do a deep dive on all of those, so don’t worry.

And it’s important to understand which ones are adapted to which situation for the exam, because the exam will definitely ask you questions to choose the right level of encryption based on the scenario. So let’s do a deep dive first on SSE Three. This is an encryption where the keys used to encrypt the data are handled and managed by Amazon S Three. The object is going to be encrypted server side. SSE means Server Side Encryption, and the type of encryption is AES 256, which is an algorithm. So for this to upload an object and set the SSE encryption, you must set a header called Exams Server Side Encryption AES 256. Exams stands for X Amazon. So x Amazon Server Side Encryption AES 256 and this is how you remember the name of the header.

So let’s have a look. We have an object and it is unencrypted. We have it right now and we want to upload it into Amazon Astray and perform some SSEs Three encryption. So for this, we’re going to upload the object onto Amazon Astray. You can use the Http protocol or the Https protocol, and you can add the header that we said, the Exams service, that encryption AES 256. And then Amazon Sere, thanks to this header, knows that it should apply its own S Three managed data key. And using the S Three managed data key and the object, some encryption will happen and the object will be stored encrypted into your Amazon S Three bucket. Very simple. But here in this instance, the data key is entirely owned and managed by Amazon Sri. Next SSE. Kms. So we haven’t seen what Kms is right now. We’ll see this pretty much towards the end of this course on the security side. But Kms is Key Management service, which is an encryption service for you.

So SSD Kms is when you have your encryption keys that are handled and managed by the Kms service. Why would you use Kms over SSCs? Ray well, it gives you control over who has access to what keys and also gives you an audit trail, each object is going to be again encrypted server side. And for this to work, we must set the header x Amazon Server side Encryption to be of value AWS Kms. So the idea is exactly the same because it is server side encryption. We have the object, we upload it using Https and the header. And then using this header, Amazon is three knows to apply the Kms Customer Master key you have defined on top of it and using this Customer Master key. So the key is defined and your object there’s some encryption that will happen and the file will be stored in your Avocet under the SSE Kms encryption scheme.

Next we have SSEC that stands for Service Head Encryption using the keys that you provide yourself outside of AWS. So in this case, Amazon Is Free does not store the encryption key you provide. So it will absolutely have to use it because it needs to do encryption, but then that key will be discarded. For this to transmit the data into AWS, you must use Https because you’re going to send a secret to AWS. And so you must have encryption in transit. Encryption key must be provided in the Http headers for every Http request made because it’s going to be discarded every single time. So we have the object and we want to have it encrypted in Amazon Astray, but we want to provide ourselves the client side data key to perform the encryption. So we send both these things over Https. So it’s an encrypted connection between you, the client and Amazon Is Free. And the data key is in the header. So therefore Amazon is tray received the exact same object and the client provided data key. And then again it is server side encryption. So Amazon Is Free will perform the encryption using these two things and store the encrypted object into your S Three buckets.

If you wanted to retrieve that file from Amazon straight using SSCC, you would need to provide as well the same client side data key that was used. So it requires a lot more management on your end because you manage the data keys and Amazon or AWS in general does not know which data keys you have used. So it’s a bit more involved. Okay. And then finally client side encryption. So it is when the client so you encrypt the object before uploading it into Amazon astray some client libraries can help you do this. For example, the Amazon S Three Encryption Client is a way to perform that client side encryption. And as I said, clients must encrypt the data before sending it to Astray.

And then in case you receive data that is encrypted using client side encryption, also CSE, then you are solely responsible for decrypting the data yourself as well. So you need to make sure you have the right key available. So as I said, in client side encryption, the customer manages entirely the keys and the encryption cycle. So let’s have an example. Amazon is three. This time is just a bucket. It’s not doing any encryption for us because it is client side encryption, not service side encryption. And so in the client, we’ll use an encryption SDK.

For example, the S three encryption SDK will provide the object and our client side data key. The encryption will happen client side. So the object is going to be fully encrypted on the client side. And then we are going to just upload that already encrypted object into Amazon S three. Okay? So that’s the four types of encryption. Hopefully that makes sense. And I’ve been mentioning encryption and transit in this lecture, and I’ll make it very clear what it is that’s around SSL and TLS connections. So Amazon is three is an Http service and it exposes an Http endpoint that is not encrypted, and it exposes an Https endpoint, which is encrypted and provides what’s called encryption in flight, which relies on SSL and TLS certificates. So you’re afraid to use the endpoints you want, but if you use the console, for example, you would be using Https.

And most clients would, by the way, use Https endpoints by default. And so if you’re using Https, that means that the data transfer between your client and Amazon Sree is going to be fully encrypted. And that’s what’s called encryption in transit. And one thing to know is that in case you’re using SSEC, so serverside encryption and the key is provided by your clients, then Https is mandatory. Also, as I said, sorry, it’s fairly encryption in Flight is also called Ssltls because it uses SSL and TLS certificates. So that let’s go into the hands on to see how encryption works now. Okay, so I am back in my bucket, and what I want to show you is the encryption settings.

So if I click on my coffee JPEG right now and we go on the right hand side, there is this encryption here. And right now it says encryption none. So this file right here, coffee JPEG is not encrypted. So let’s go see how we can specify the encryption options. So I’m going to upload it, add a file, and this time I’m going to use the same coffee JPEG. And when you click next, the permissions, I’m going to leave everything as is, the properties everything as is. But now on the encryption side, so scroll down. There are three options. There’s none, which is no encryption, and there is Amazon S three master key and Amazon AWS Kms master key. So this one is SSEs three, and this one is SSE Kms. So if we choose SSE three, then this is just one option. We don’t have to provide any key, but if we use SSE Kms, then we have to select a Kms key. And by default, there’s going to be a service CM case. So AWS Estrade, which is a key dedicated to Amazon as ray that we could use, or we could provide our own custom Kms key, in which case we would have to create that key in advance.

 And this is something we haven’t done yet. We’ll do this in the Kms section, but this is just to show you the two different options. So in this example, I’m just going to use Amazon’s three master key, click on Next and finally Upload. And now, as we can see, my coffee has been uploaded. I click on it and on the right hand side it says Encryption AES 256. So, perfect, this has been encrypted. If I go to Versions Show again, we have all the versions of our objects, and we can see the latest version of our coffee is encrypted, but the version prior to that is not encrypted.

So the encryption really belongs to the object you are uploaded during the operation. Okay, so let’s go back to Versions hide. I’m going to upload my beach JPEG. So I’m going to take this beach JPEG in here and I’m going to go Next, next, and then in the encryption, I’m going to choose Kms and I will choose AWS S three and click on Upload. And here we go. So now my beach JPEG also has been uploaded. And this time the encryption scheme is AWS Kms. So, perfect, everything is working. Now, last thing I want to show you is this concept of default encryption. So, if I go to Properties and scroll down, I have a default encryption, which is to automatically encrypt objects when stored in Amazon Straight. So that means that if you are uploading an object and you don’t specify an encryption scheme, we get three options. Either we leave it unencrypted or we by default apply AES 256 or we by default apply AWS Kms. So, for example, we can say, okay, I want to automatically encrypt all my objects with SSCs three if they come unencrypted.

Okay? So I click on Save, and now by default if we upload an object. So I’m going to upload an object at a file and I will choose again my coffee JPEG. And this time I’m not going to set any encryption. So none and click on Upload. We would expect it to be none encrypted. But if I click on Coffee JPEG, we can see the encryption is AES 256, because by default, thanks to this default encryption, it has been automatically encrypted. Okay, now finally you may ask me, Stephan, why are we seeing only three options, none as 256 and AWS Kms? Didn’t you say to us that there was four security options? The answer is yes, there are four encryption options, but the other one is SSEC, and we would have to provide our own key for the encryption. And this is an option that AWS chose not to implement in their console. So we cannot do SSEC through the AWS console, but we can do it using the command line interface, and we’ll see what the command line interface is in a few lectures.

And the last option is client side encryption. And so for client side encryption, this is us, the client that should upload and encrypt the object already encrypted. So upload the object already encrypted. And so therefore, we have no option here as well. So these options make sense. Okay, so that’s it for encryption here we have a cool combo of both the versioning and encryption, and we’ve seen the concept of default encryption, so I hope that was helpful, and I will see you in the next lecture.

5. S3 Security & Bucket Policies

Okay, so now let’s talk about Amazon stray security. So it’s very complex, but first you have user base security. So, our im users have im policies and they authorize which API calls should be allowed. And if our user is authorized through im policy how to access our Amazon tree bucket, then it’s going to be able to do it. Then we have resourced based security. And this is infamous S Three buckets policies. They’re bucket wide rules that we can set in the S Three console. And what they do is that they will say what principles can and cannot do on our S Three buckets. And this enables us to do cross account access to our S Three buckets. We’ll do in the hands on a very deep dive on S Three bucket policies. Then we have object ACL, which is finer grain where we set at the object level the access rule. And then finally bucket ACL, even less common.

And these two don’t really come up at the exam. Okay, note an in principle. So it could be a user, a role can access an S Three object if the im permissions allow it. So that means that you have an im policy attached to that principle that allows access to your S Three buckets. Or if the resource policy. So usually your S Three buckets policy allows it and you need to make sure there is no explicit deny. So if your user through im is allowed to access your S Three buckets, but your bucket policy is explicitly denying your user to access it, then you will not be able to access it. Okay? Okay, so now as you deep dive on S Three bucket policies, they’re JSON based policy.

So JSON is a notation language. And so we have here a JSON bucket policy. And this bucket policy here allows public read on our S Three buckets. So as we can see, it says effect allow principle star. So anyone the action get objects on the resource example bucket star so on any objects within my S Three bucket. So this is great. This allows public access to our Avocet.

So these bucket policies can be applied to your buckets and objects. So both the actions is they allow a set of API to allow or deny. The effect is allow or deny. The principle is the account or the user that this SRE bucket policy applies to. And so some common use cases for SF bucket policies is to grant public access to a bucket or to force objects to be encrypted at the upload time, or to grant access to another account using cross accounts sir bucket policies. So we’ll do in the hands on a deep dive on Sri bucket policies. Then we have the bucket settings for block public access.

So we’ve seen this in the hands on when we get started. So this was a new setting that was created to block objects from being public. If the account had some restrictions. So here we have four different kinds of block public access settings. We have the new access control list. Any access control list or new public or access point policy. So this is going to block objects and buckets from becoming public if they’re granted through any of these methods.

Or you can block public and cross account access to buckets and objects through any public bucket or Access Point policy. So you don’t need to remember these four different settings, okay? It’s just a summary in here. What you need to remember going into the exam is that there is a way to block public access to your S Three buckets through these settings. The exam will not test you on each of these settings, okay? These settings historically were created to prevent company data leaks because there were a lot of leaks of Amazon S Three buckets in the news. And Amazon S Three came up with this way of making sure that an administrator could say hey, none of my buckets are public by the way, because of these settings. And that was very popular.

And so if you know that your buckets should never ever be public, leave these on. And there’s a way to set these at the account level as we’ll see in the hands on other securities in S Three you should know about on the networking side, you can access S Three privately through VPC Endpoints. So if you have easy to instances in your VPC without Internet access, then they can access S Three privately through what’s called the VPC Endpoints. For logging audits, you can use S Three access logs and they can be stored in the other S Three buckets. API calls can also be logged into Cloud Trail which is a service to log API calls in your accounts. For user security you have MFA delete.

So multifactor authentication is MFA, in which case if you want to delete a specific version objects in your bucket, then you can enable MFA delete. And we will need to be authenticated with MFA to be able to delete that object. And finally, precinct URLs that we’ve seen briefly when we were opening that file and there was a very long URL, which is a URL that’s signed with some credentials from AWS and it’s valid only for a limited time.

And the use case for it, for example, is to download a premium video from a service if the user is logged in and has purchased that video. So the idea here is that any time at the exam you see the access of certain files to certain users for a limited amount of time. Think pre signed URLs. So in the next lecture we’ll do a hands on espresso security to see all these various options. So I will see you in the next lecture.

6. S3 Bucket Policies Hands On

Okay, so let’s have a look at the different security settings for our S Three buckets. So the first one I want to show you is going to be around the S Three Bucket Policy. So if you go sorry to permissions and then you go to Bucket Policy here we are able to set a Bucket policy for our S Three buckets. And so we have to put the JSON document in here and for this we can use these documentation to get some sample policies or use the Policy Generator. So we’ll use the Policy Generator because it is a cool exercise, you don’t need to remember how to use it exactly, but I think it’s a cool exercise and we’re going to design an Amazon Policy Bucket Policy that will prevent us from uploading an object unencrypted without the AES 256 encryption mode. So the select type of policy is going to be an S Three Bucket Policy and then the effect is going to be deny. We will deny Principal Star which means anyone. So we’ll deny anyone on the service Amazon straight for the action.

So we’re talking about upload. So the action is called put objects. So to Upload we look for put objects. So we deny anyone on Amazon for the action put Objects and then the Amazon resource name or ARN. You’ll see this a lot in this course is a format like this and to get the ARN we need to go back to S Three and we get the ARN right here. So we copy all this ARN. It starts with ARN column. So we paste it here. And at the end of this, because this is a put object that we apply it to, so Put object, it needs to be applied to anything within the bucket. So at the end of the ARN we need to add a slash star and this represents any object within the bucket. This is why we add a slush and a star and we add the statement. And first we need to add conditions obviously. So we’re saying we deny anyone on Amazon a Three for put objects on any object within our buckets. And the first thing is that we want to make sure that there is a header for encryption.

So we’re looking for null and the key is going to be the encryption header. So here we go, we’re going to find it here, amazon Server side Encryption. So n the value is true. So we’re saying here if this Ximz server side encryption header is null, so is null, that means it doesn’t exist then deny it. If it’s null, that means that the object is sent unencrypted. So this is what we want. So we’ll add the statements. This is the first statement in here and then we’ll add a second statement. So we’ll say principal star effect deny. Again we’ll find the Put object in here. So put objects. And for the resource name we’ll have star. So just same as before. And then we’ll add a condition and we’re saying okay, string not equal. So this is another condition and so we’ll look at the same object header. So we’ll look at the AMC server side encryption and we’re saying, okay, if it’s not equal to AES 256, which is the encryption used when we have SSEs three, then deny it.

So through this second statement here, what we’re doing is that we’re denying any request that does contain this header but is not equal to as 256. We’ll add the statements. Now we have two deny statements and hopefully that works. We’ll generate the policy and the conditions did not work. So let’s go back and edit this real quick. So I think the problem I had was around the fact that did not add the condition. So let me quickly fix this. So star deny and it’s good to keep mistakes on camera sometimes because this is things that you can go through as well on your own. So no one is perfect. So I’ll just quickly do this. So okay, and I’ll add a condition and we’ll do this time again, null. We’ll do it twice, I’ll show it once and then I’ll fix it twice. So Amazon x three server side encryption value true. And here what I was missing is to click on add condition. So here we go. Now this condition has been added, okay, so this looks good, add statement. And the second thing is deny star, amazon is free again, looking for put objects, here we go on the same star add condition and we’re saying string not equals same header here. And finally the value is a yes, 256.

Okay, here we go, add condition and add a statement. Okay, so we have a correct policy. So I’ll generate the policy and this time it looks good. So I’m going to go ahead and copy this and paste it in here and save it. So now we have a bucket policy and we need to see if it works. So for this, no better way than just test it out. So we’re going to back going back to our bucket, we’re going to upload a file and this time we’re going to upload our coffee JPEG. Click on next, next and I will make sure to have encryption none. Next and upload. And it’s starting and we get one error. So here there was an error and so if we view the details of the error we’re saying it’s a forbidden error and if we look at it, it was forbidden because it did not include the correct header. So we had no encryption and this was denied by the S three bucket policy. Now if you upload the very same file coffee JPEG, but this time we do specify the as 256. So by using Amazon’s free master key upload.

Now this time it should work. So yes, this was a success and finally if we go to Upload, we add the exact same coffee that JPEG. But this time we specify the SSC Kms encryption scheme. Click on next and upload. This one should give us an error. Yes, it did give us an errors. Now we have two errors. So as we can see, this S three bucket policy is active and is working. So obviously I did not remember how to do this bucket policy on my own. Okay, you can just type S three bucket policy, deny encryption on Google and you will find some blogs, for example, that show you some bucket policies that allow you, for example, here it is that allow you to, for example, deny encryption incorrect encryption error or deny unencrypted object uploads. So these are things you can find online. But I thought it was really cool to show you this address policy generator, which I think is a really cool tool to create policies. Okay, back into Sree. We also have the access control list, which to be honest are not at the exam anymore. So I’m not going to linger on those. But this is where we can set access control list at the bucket level, or we can set access control list as well on this object.

So if I click on the object and go to permissions, then I can set the ACL at the object level. But again, I’m not going to linger on it because this is out of scope for the exam. And then finally, if we go to permissions, block public access. These are the four settings that I was mentioning to you. They’re described right here. And right now everything is on. So it’s blocking all public access. But if you wanted to make your bucket public, then the first thing you would need to do is to untick everything and click on Save, which I’m not going to do right now.

Okay, but you see where these settings are, and again, don’t remember the name of the four settings. Just remember that these settings exist and that there is a separate level of protection for your S three buckets. And as I said, these settings can also be applied at the account level. So if you’re on the left hand side, you can have the account setting to block public access. When you know that all the Svckets in your account should not be public, then you should enable it here as an extra security. Okay, that’s it for this lecture. I hope you liked it. I’m going back in my bucket and I will see you in the next lecture.