Amazon AWS SysOps – S3 Fundamentals

1. S3 Fundamentals Section Introduction

Welcome to the first part of the S three section. Now, s three is a huge exempt topic and it is so important to master it. As you can see, there are a lot of technologies within the S three box. So I’ve divided this section in actually two parts. The first part is going to be around the S Three fundamentals and that comes straight from the developer course. You’re free to just watch all the lectures if you need a refresher. That includes buckets and objects, versioning security websites, cores and consistency model, especially around versioning encryption and bucket policies. You need to know the basics for sure. Now, if you know already all these things in and out, feel free to skip this section and get back to the second section where I deal with S Three, especially for sys ups. And trust me, there’s a lot of things to learn there as well. I hope you’re excited. Let’s get started.

2. S3 Buckets and Objects

Okay? So first to talk about Amazon S Three, we need to talk about buckets. So S Three is a system, a service that allows us to store objects. So files into buckets or directories. And each bucket must have a globally unique name. As you’ll see, in the hands on, we can’t create a bucket that has already been taken in terms of names. The buckets are defined at the region level. So even though S Three is a global mobile service, buckets are a regional resource. And there is a naming convention which includes no uppercase, no underscore three to 63 characters long, it should not be an IP, and it must start with a lowercase letter or a number. Okay? Now in these S Three buckets, we need to create objects. And objects are files and they must have a key.

And what is a key? It is the full path to that file. So if we have a bucket named my bucket and object named my file. TXT, then the key is my underscore file TXT, it’s the blue part. But if we have folder structures within our three buckets, so my. Underscore folder one another folder, and then my file. TXT, the key is the full path. So that’s again, the entire thing in blue. And the key can be decomposed in two things, the key prefix and the object name. So if we take the same long example, the prefix for myfile. TXT is my underscore folder one another folder. Okay? That is the prefix, whereas the object name is my underscore file TXT. So even though there’s no concept of directories within buckets, just very, very long key names, then the UI will try to trick you into thinking otherwise, because we could create directories within Sri. So what we really have in history is just keys with very long names that contain slashes. Okay? So now let’s go again for this object.

So the object values are the content of the body. So a maximum object size on Amazon history is five terabytes. So 5000gb, which is a huge object. But you cannot upload more than five gigabyte at a time. So that means that if you want to upload a big object of five terabytes, you must divide that object into parts of less than 5GB and upload these parts independently into what’s called a multipart upload. Now each object in Amazon history can have metadata. So list of key value pairs that could be system or user metadata. And this is to add information onto your objects and also tags. You can also have key value pair and tags, which is very useful when you want to have security on your objects or lifecycle policies. Finally, we’ll see when we go into versioning that there is a version ID onto our Amazon S Three budget and we’ll see what the value of that is in the versioning lectures.

So without further ado, let’s get into the Amazon S Three console and do a hands-on. OK, so let’s do our first hands-on on Amazon history. So I’m just going to type S Three in the search bar, and we are going to be redirected to the Amazon S Three console. So as we can see, there are improvements to the console. So hopefully yours looks like mine, but if there is a big, big change, I will update this course, obviously. Okay, so the first thing we can see in this UI is that the top right corner says global. So S Three does not require region selection. So that makes S Three a global service. So in this UI, you will see all your buckets. But as I said, buckets are created for a specific region. So when we go and create a bucket, we will still have to select a region. So this global view will give us all the buckets across all the regions. But a bucket is tied to a specific region. So let’s go ahead.

I’m going to remove the panel on the left hand side and I’m going to go ahead and create a bucket. So I have to enter a bucket name, and the bucket name must be globally unique. So for example, if I type test in here, this bucket name should be taken because probably someone already thought about creating a bucket name of test. So it’s going to create the bucket and it says, yes, the bucket with the same name already exists. So it doesn’t matter if this bucket doesn’t exist yet in my account, it exists in someone’s account, and so it’s already taken.

So instead of going to say the bucket of Stefan 2020 and that bucket name should be not taken and we’re good to go, then we have to select a region. So in this instance, I select EU Ireland US One because it’s one that’s close to me. But you should select a region that’s close to you as well to create your S Three buckets. And as you can see, not all the regions have S Three just yet. Okay, so select the region close to you, and we can see that this bucket is going to be tied to this region, even though it will show up as a global service. Okay, so let’s scroll down and see all the settings. So there’s bucket settings for block public access.

And I will leave this on. Right now. We’ll have a deep dive on this. This is to prevent your bucket from being public by mistake. So for now, we’ll block all public access and keep our bucket as private. And this is to prevent data leaks that can happen in the news. So this was a setting that was added later last year or two years ago to really prevent buckets from going public mistakenly. The advanced settings, we will get to see them all one by one in the future. Lectures. So what I’m going to do right now is just take these buckets and click on Create Buckets. Okay, so my bucket has been created and now I can go to the bucket details either by clicking this link or by clicking the link in the console as well. So now what I can do is to go ahead and upload a file.

So for this, I’m going to the Upload button on the top left and then click on Add File. I’m going to select Coffee JPEG and upload that coffee file. So as you can see, we can directly click on Upload here or we can go and go through some options. So let’s just quickly view these options even though we won’t touch them right now. So I click on Next and here we have some information about the permissions of who can see that file. So my account can read and write to this file. So that’s perfect. And for now, we cannot manage public permissions because it has been blocked by the settings we submit from. Before I click on Next, we can look at Properties, which is depending on how we want to store that object for a storage class. And we’ll have a deep dive on this as well. Encryption will have a deep dive on this, metadata and Tags. For now, I’m going to leave everything as default and click on Next.

So overall, we haven’t changed anything. I just want to quickly glance through the options and I’m going to go and click on Upload. Now my file is being uploaded onto my extra bucket. And here it is. Perfect. So my file has been uploaded, I can click on it and there is a panel on the right hand side that comes up and it gives us some information about our file. So let’s go ahead and do something. I’m going to right click on this file and then click on Open. So through this way I am able to view my coffee JPEG file and as you can see, it is the image that I’ve uploaded. So this works, but the URL, as you can see, looks a bit long and very complicated. If I go the other way, click on this file and click on the object URL from this panel. So I’m going to click on the object URL here and go back to my file sorry, and open this new tab. It says the XML file does not appear. If you have any style, it says Access denied. So somehow using this URL, which is a public URL for my file in history, I get a 403 access denied. Whereas using this URL, by doing right click and Open, we are able to access this file.

So the reason is this file is not public. So anytime we try to access this file publicly, because for now we haven’t uploaded and updated our bucket to be public, we will not be able to access it. But if we use the right click and Open, it actually creates a special URL. And this is a very special URL that’s very, very long. And this URL is called the presigned URL. It’s signed with my credentials on AWS. And so because I have the power to view this file, it’s going to be included in that URL and I’m going to be able to access this file. We’ll have a deep dive look at presigned URL anyway, but just to give you an overview, so for now, we’ve seen that the public version of our file is not accessible because our bucket is private and that we’re able to access our file privately using the presigned URL.

What I can do as well is create a folder, so I can create a folder called Images and click on Save. And then within this folder I can go ahead again to upload my coffee JPEG file. And now my file has been uploaded yet again. And so, as we can see now, if we look at our SD bucket, we have two keys. We have the coffee JPEG at the root of our SD buckets and we also have a coffee JPEG within the key images. So the path images. So we have different keys and it gives us the illusion to have a directory, but it’s actually just a very long name for our file, coffee JPEG.

So this is the entire file name. So we have the same file in two different buckets into different path. Sorry. And what we can do as well is to do operations on our file. So for example, we can rename the file or delete the file copy move, so we can do a lot of operations on our files. So what I’m going to do, just to show you is go to this directory here and I’m going to do a delete and this is going to delete my image within the images directory. So I click on delete and here we go. It’s gone. And so that’s just the basics of s three buckets. We have a lot more to see as you can see, a lot more options and so on, but hopefully you liked it and I will see you in the next lecture.

3. S3 Versioning – Basics

So now let’s talk about Amazon’s three versioning. So your files in Amazon is three can be versioned but it has to be enabled first at the bucket level. So we’ll do this in the hands on. So that means that if you re upload a file version with the same key then it will override it but it won’t override it actually it will create a new version of that file. So instead of overriding the file file that already exists it will create a new file version and I’m simplifying it here but it will be version one, then version two, then version three, et cetera, et cetera. So it is best practice to version your buckets in Amazon stray in order to be able to have all the file versions for a while because you can get protected against unintended deletes because you’re able to restore a previous version and also you can easily roll back to any previous versions you wanted.

So you need to know a few things though. Any files that is not versioned prior to enabling versioning will have the version null and if you suspend versioning in your bucket it does not delete the previous versions, it will just make sure that the future files do not have a version assigned to it. So let’s go in the hands on and see how that works. So here I am in my bucket and I’m going to go to the properties and in properties I have versioning so I’m going to click on versioning and click on enable versioning and finally save. So this is to keep multiple versions of an object in the same bucket back in overview and now we have a new panel called versions and you need to click on it.

So right now it says hide but I want to go into version show and now we have a different UI into Amazon s three console which is showing the version ID on top of the file name. So for this coffee JPEG the version ID is null because we are enabling versioning after having uploaded a few files as I said in the theory lecture then this will have a version ID null but let’s go ahead and upload a new file. So I’m going to add a file and this time I’m going to upload a beach JPEG and upload this file and what we get out of it is that this beach JPEG has a version ID and it’s not 1234. I was making it very simple here.

The version ID is a much more complicated string but as you can see there is a version here and it says latest version so I can go ahead and re upload for example that beach file. So let’s go ahead and upload that beach file and as we can see now it is successful and now we have two versions of that file beach JPEG, we have the previous one and then we have the latest version which was uploaded at 02:18. P. m. Which has a different version ID.

Even though it was the same file, it was not overwritten. It has been uploaded as a new file version. So we can do the same with the coffee. So I can upload, for example, the coffee JPEG and upload it. And as we can see now there is a latest version for coffee JPEG with a version ID and even the file from before that had the version ID null is still kept in here. Okay, but if we go back to version hide, we only see two files. So let’s do something quite interesting. We’re doing versions hide and I’m going to take my beach JPEG right click and then I want to delete it. So I’ll say delete and click on delete. So now it seems that my file is gone. We cannot see it in this window, but if we go back to versions show, we can see that the beach JPEG is still here. And the thing that gets added for beach JPEG is a delete marker. So a delete marker has a file with zero size that has a version ID.

And because it is the latest thing on my beach JPEG, this delete marker made my file be hidden. But what I can do is always restore a previous version. So if I can click on this delete marker and I delete my delete marker, what’s going to happen is that now the latest version is one of the previous versions and if I go to here, my beach JPEG is back. So this is what I mean by preventing against unintended delete. If somehow I go in this beach JPEG and unintended do a delete of this, then it’s still going to show because it was just a delete marker.

Now it is possible for you to remove a specific version ID of a file. For example, if I go to this one right click and then do delete, then this will delete this specific version ID in a stream bucket and this will for sure delete that version ID permanently. So it’s very important to understand that. And so that’s all there is to know about versioning. Obviously you can go and suspend the versioning. So if I go and suspend the versioning, then any previous objects are still here and they still have a version ID. It’s just affecting future version objects.

Okay, so I’m going to go back in versioning and re enable it. But you can play around, see how that works. But versioning is a very powerful feature of Amazon astray because now you can keep multiple versions of a file for a long time and ensure you can roll back or restore any previous versions if you want. That’s it for this lecture. I hope you liked it and I will see you in the next lecture.