Amazon AWS SysOps – EC2 Storage and Data Management – EBS and EFS Part 3

9. EBS Operation: Volume Migration

All right, finally, let’s talk about the last operation, which is, how do you encrypt an EVs volume? So, when you created an encrypted EVs volume, right away you get the following. You get data at rest being encrypted inside your volume. All the data inflight between the instance and the volume is encrypted. All the snapshots will be encrypted, and all the volumes created from the snapshots are encrypted. So there’s encryption all around the place. And the old encryption and decryption mechanism is handled transparently for you, so you have nothing to do. It’s all handled by EC two and EBS behind the scenes. So encryption overall is something you should use because it has a very, very minimal impact on latency, almost nothing. And it leverages keys from Kms.

So AES 256, that’s something that you should know. And so when you copy an unencrypted snapshot, then you enable encryption. So let’s talk about a very important thing, which is how do you encrypt an unencrypted EBS volumes? So, to encrypt an unencrypted EBS volumes, which is very tough thing to say, you create an EBS snapshot of the volume. Then you encrypt the EBS snapshot using the copy function. Then we create a new EBS volume from the snapshots, and that volume will also be encrypted. And now we can attach the encrypted volume to the original instance.

So let’s go have a look at how we do this in the console. Okay, so if we look at our volumes, this one, that was the one we created from before. The encrypted flag says not encrypted. So this volume is not encrypted. And maybe we wanted to encrypt it. So we right click, we create a snapshot. But we already done that. So we go to the snapshot, and here is our snapshots. As we can see, our snapshot encrypted is not encrypted. So because we did a snapshot of a non encrypted EBS volume, we get a non encrypted snapshots. Now, right click. If I wanted to create a volume from it, as you can see, the volume will also be not encrypted. So it’s not what we want.

So, as I said in the lecture, what I need to do is right click and copy. By copying the snapshots, I can click here and encrypt the snapshots. So I can put any region that I want, but I can stay within the same region, which I will do, and I will say, okay, encrypt the snapshot using the default AWS EBS master key copy. And here we go. Now the snapshot is being copied. And if I refresh now, we can see that we have a copied snapshot right away from here. So the snapshot is being created. Now, the snapshot, if we look at the bottom right, it says encrypted, encrypted. And then we get some Kms information around how it’s been encrypted, which is quite neat.

So let me just wait for the encryption copy to happen. And so now it’s completed I right click and create a volume from it. And here we go. Now, the encryption is encrypted. And so if you were to create a volume of 5GB and maybe EU West One B, just to keep the same AZ and click on create Volume and go to that volume now in the Volume EBS thing. So I’ll just wait just a little bit. Here we go. Here it is. Now, we can see that this new volume in EU West One B is available, and it is encrypted right away for us right off the bat. So this is pretty cool. This is how you would go and encrypt an EBS volume. I know it’s very manual, so you could automate this if you wanted to, but super important for you to see once. Okay. I hope you enjoyed it. I will see you in the next lecture.

10. EBS Operation: Volume Encryption

So let’s talk about the difference between an EBS Volume and an Instant Store. So basically, some instances do not come with root EBS volumes. They just come with something called Instant Store. And instant store is equivalent to ephemeral storage. So what’s the difference between Instant Store and EBS volumes? Well, Instant Stores is something that is physically attached to your hardware, whereas EBS was a network drive. And so imagine that the big racks inside of Amazon’s data centers, some of these machines, some of these EC two instances we get, they will have a physically attached disk, and that will be an instance store. And the idea is that why would we even use this? Well, because there is no network. You’re going to get better IO performance.

You’re going to get a good utilization. If you want it to have a buffer, a cache, you want to scratch it out, and you want to store temporary content. And if you wanted to, for example, reboot your instance, that’s fine. The data will survive your reboot. But why would you not use an Instant Store? Well, on stop or termination, your entire instance store data is lost. So it’s not very good. And you can’t resize the instance store on the fly just the way we did from EBS volumes. And if there’s any backups you need to do, you can’t just, like, right click and backup. You need to do this yourself. So Instant Stores do have a big use case for caches or whatever, but they’re not for every different kind of use case. So if in the exam, they ask you, should we use an EBS Volume or Instant Store?

Ask yourself, am I okay losing my data, or is my data ephemeral? If not, use EBS? Just want to reiterate what I already told you, but it’s very important that I make sure you have this across perfectly. So the instance store is a physical disk attached to the physical server where your EC Two is. And it’s very high IOPS because it’s physically attached. Okay? EBS volumes were network based, and so there was networking between your instance, and so there was limited IOPS. But with EC Two instance or it’s really, really high I ops. Here is an example from the documentation. If we consider, for example, an I three eight x large, we’re talking about 720,000 write I ups and 1. 65 1. 65 million read I ups.

So this is huge. And obviously it can go even higher to 2 million, 3 million IOPS on the read and 1. 4 million on the right. So when you’re in the exam, you think about high, high, high IOPS, maybe hundreds of thousands of IOPS. You have to think local EC Two instance store EBS cannot achieve such IOPS for you. Okay? They’re limited up to 64,000 IOPS, and that’s for IO one. So the disk, though, they’re up to 7. 5 terabytes. That can change over time. So maybe as a plus adds more instances, they will be bigger and they can be striped to reach all the way to 30 terabytes and then get that number. It can change over time, but once you set up a disk in local instance or it cannot change its size. It’s also block storage so you can have a file system on it just like EBS.

So from your instance standpoint it’s still a file system, it’s still just a disk. Okay? The difference is that it’s a physical disk, not a network disk. And so, as I said, yes, it cannot be increased in size. So once you provision your local easy to instance, or you cannot resize it over time or add new ones, whereas for EBS you could change its size over time. And finally I’ve told you this again, but I’ll tell you it again there’s a risk of data loss if your hardware fails. That means that if you stop your instance and you lose it, then you’ve lost the physical disk attached to it and you lost your data.

So make sure you don’t store very sensible data on your Instant Store as long as it’s not replicated, okay? Make sure that if you use an Instant Store you’re going to replicate your data across other instance sellers in other instances to have some kind of redundancy. Okay? So if I wanted to go ahead and launch an instance from there I’ll choose Amazon Linux Two. I’ll select this and in terms of the instance type I will choose something that has Instance Store. So as we can see, for example, C 5d Large will have an instance store.

So I will click on this, click on configure instance details. Click on Add storage. And as you can see now we have a root volume right here which is still an EBS volume. But then the volume type we have here is called Ephemeral Zero. So it’s instance store so it’s going to be named Ephemeral and we have a 50 gigabyte instance store that we can use. And the really cool thing about it is that it’s going to be high performance for this 50 gigabyte. So we could place some cache data, for example, there. So this is what I want to show you. I’m not going to go ahead and create that instance because there’s no point and I’ll spend money on it. But just so you see that to create an instant store backed, easy to instance, you have to choose some kind of specific type of instance. So hope you liked, I hope you enjoyed it and I will see you in the next lecture.

11. EBS vs Instance Store

So let’s quickly talk about EBS for SysOps. So if you plan to use the root volume of your instance after it’s being terminated then you need to set the delete on termination flag to no and remember that was this tick box that we could take when we created our instance. And you can see that as I said, this option when we create our instance. Yes. So if you use EBS for high performance then you need to use an EBS optimized instance type, otherwise wise you’re not going to get the maximum throughput that you’re hoping to get. So to look up which instances are EBS optimized, you can go on the AIS website and they will let you know if an EBS volume is unused, you’re still going to pay for it.

So be careful, don’t over provision, use what you need and then you can resize up. So for cost saving though, if you have an unused EBS volume, what you can do is create a snapshot of it and restore it later and that will basically give you three X cost savings. So it’s quite nice. Finally, if you get to high wait time or slow responses for your SSD drive you need to increase the IOPS on your GP two or move to Pi ups provision IUPs on IO one, EC two will not start if your EBS volume is not mapped correctly.

So for example, if you have a Dev Xvdb volume instead of Dev xvda, this is going could be a reason why your EC two won’t start. And then after increasing the volume size, you still need to repartition the volume as we saw before, just to make sure that you can use the incremental storage. So if you were to use XFS, the command will be called XFS grow FS. So that’s it just the little nits you need to know as the sys apps for the exam. And I will see you in the next lecture.

12. EBS for SysOps

So something sysaps needs to know how to do is to do EBS migration. So, as I said, EBS volumes, they’re only locked to a specific AZ. And so if you wanted to migrate it to a different AZ or a different region, you need to snapshot the volume, which you just did. Then we need to optionally copy the volume to a different region, and then we could create a volume from the snapshot in the AZ of our choice. Very simple, but it’s good to see it once. So let’s practice. So, as a reminder, finder if I go to my volumes, both of my volumes are in EU West One B. But say there was an instance in EUs One A that I wanted to attach this volume to. What I wanted to do is go to snapshots, right click, create a volume.

And here I’ll say, okay, it’s a 5GB GP two volume. Great. And now the AZ I want to put it in is EU West one A? But I can choose whatever I want. So now I’ll just say create volume. And here we go. Now we go back to our volumes, and very quickly, as we can see, we’ll get a new available volume right away. Here 5GB restored from a snapshot, as we can see. So it’s a snapshot, and it just points from my snapshot, and it’s available in UAE West One A. And so that’s perfect. We have very quickly, using snapshot, migrated a volume from EU West One B to EU West One. Very simple thing to do, to be honest. But you need to see it once as a sys ups. So I hope you enjoyed it, and I will see you in the next lecture.

13. EBS RAID configurations

All right, just a few more things you need to know about EBS. So EBS has a Raid option, and so Raid is usually traditionally used with disk on your own data center. But EBS is a bit special because EBS is already some sort of redundant storage. It’s already replicated within your AZ. But if you wanted to increase your eye up to say, 100,000 IUPs, how would you do that? Or what if you wanted to mirror your EBS as volumes because you didn’t trust Amazon for making your storage redundant? What if you wanted to mount volumes in parallel using rate settings? And that’s something you can do. And so Raid will be possible as long as your operating system will support it.

So that means Linux or Windows, and you have many Raid options out there. There’s rate zero, rate one, and these are the ones that they will ask you about at the exam. But there’s also rate five, which is not recommended for ABS. You have to see the documentation Y and rate six, which is also not recommended for ABST documentation. And so also rate ten. But we’re not talking about it. So in this lecture we’re just going to focus on an introductory to rate zero and rate one for EBS. Okay? So what is rate zero? And you need to know about it. It is a way to increase performance. So if you need to make a mind map ray, zero means performance.

So we have our C two instance and it has one logical volume. But that volume is a bit special because it is backed by two or more EBS volumes. So we have EBS volume one and EBS volume two in this example. And so when you do a write, it’s either going to EBS volume one or it will be going to EBS volume two. So when you write data, for example, I’m writing blocks A-B-C and D. As you can see, they get distributed between the two volumes. And so you combine these two volumes into one logical Raid zero stripe, and what you get out of it is the total disk space and I O. So if your EBS volume one is 50GB and EBS volume two is 50GB, you get 100GB.

The idea though, is that if one of these disks will fail, you lose all your data, all your logical volume is gone. So when you have this, you increase performance, but you are risking more. You are increasing your risk to have faults. So use cases for this would be an application that will need maybe a lot of IOPS and doesn’t need fault tolerance, or maybe a database that has already replication built in and can leverage a bit of fault tolerance as well. So using this mechanism, we can have a disk with a lot of IOPS. We can go all the way to 100,000 IOPS if we wanted to, by using ten volumes with 10,000 IOPS each so here’s an example.

If you have 2500gb Amazon EBS IO one volume and each of them has 4000 provision I ups that will create a 1000GB rate zero array with an unavailable bandwidth of 8000 I ups and 1000 megabytes of throughputs. And so that’s the cool thing here. We get to see how things work and we get to see that, yes, if we do have two EBS volumes in rate zero mode, then he is going to sum up the size, the disk space and sum up the IO. Now, similarly there is Raid one and this one is to increase fault tolerance. So Raid Zero was for performance, red one is for fault tolerance. And so the diagram is kind of similar. We have any situations and has one logical volume exposed to it. But this time we have EBS volumes one and two and we are going to write to both at the same time. So anytime we write a block A on volume one, it will also go to volume two B and C.

And so they’re also going to happens from the OS site. So your EC two instance will be writing to both volumes at the same time. And so this is right one and it’s called Mirroring. So you can mirror a volume to another because volume one and volume two have the exact same data. The idea is that if one of the volume fails then our logical volume is still working because we have a copy of the data somewhere else. So we have to send the data to two EBS volumes at a time. That means that we have to use two times the network throughput. So it is something to really make sense of because if you use red one you’re going to need an easy two instance that has more network throughputs to handle the rights to two EBS volumes at a time.

Use cases for this will be an application that needs to increase volume fault tolerance and you don’t trust AWS or an application maybe where you need to service disks. So for example, if you have 250 gigabytes Amazon EBS volumes IO one with 4000 provision IOPS each, that will create a just one 5000 gigabyte right one array and it will have the exact same IOPS, 4000 IOPS and 1500 megabytes of throughputs. So the only thing that you’ve added here is fault tolerance. You haven’t added any performance to it.

So you need to really remember these two write zero and write one. And it’s not something that lives in the AWS console, it is something that you have to do on the OS, on your Linux, on your Windows. So we have to go through like some configuration. Now, in the documentation there is a way to do it and you’re you can just walk through it on your own for practice. But for the Sysaps exam they don’t expect you to know how to do it, they just expect you to know that you can use EBS volumes in rate zero or raid one setting. And here is how they work. So I hope you enjoyed it, I hope you liked it, and I will see you in the next lecture.